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SUMMARY 


Problem 


This  project  is  part  of  Advanced  Development  Objective  PU3-03X,  Educa¬ 
tion  and  Training.  The  objectives  of  the  project  are  to  develop  com¬ 
puter  assisted  instruction  (CAl)  techniques  and  methods,  to  test  CAI 
training  modules  derived  from  these  techniques  and  to  investigate  the 
feasibility  of  integrating  CAI  into  Navy  training.  This  is  the  final 
report  on  the  development  and  evaluation  of  CAI  techniques  which  were 
derived  for  a  small  dedicated  computer  system.  The  computer  system 
used  was  the  IBM  1500  Instructional  System  and  the  curriculum  was  taken 
from  Basic  Electricity/Electronics  (BE/E)  School.  The  project  included 
three  major  development  and  evaluation  areas:  (l)  instructional  strat¬ 
egies  for  CAI,  (2)  computer  based  evaluation  and  revision  methods,  and 
(3)  empirical  evaluation  of  the  effectiveness  of  CAI. 


Approach 

The  development  of  CAI  lessons  consisted  of  a  series  of  steps  beginning 
with  identification  of  training  objectives  and  construction  of  criterion 
tests  and  continuing  with  planning  of  instructional  sequences  and  strat¬ 
egies,  preparation  of  lesson  materials,  review  and  revision  of  lessons, 
coding  and  checkout,  student  tryouts,  analysis  of  lesson  deficiencies, 
revision  of  lessons,  and  operational  tests.  In  the  process  of  develop¬ 
ing  CAI  materials  a  number  of  special  branching  strategies  were  devel¬ 
oped  to  provid^  pretest  branching,  branched  remediation,  student  control 
of  instructional  sequences,  dual  control  of  practice  by  both  student  and 
program,  and  optional  reviews. 

The  analysis  of  student  performance  to  identify  lesson  deficiencies 
required  the  development  of  a  new  student  performance  data  management 
system.  A  complete  revision  methodology  was  also  developed  to  assist  in 
the  revision  process. 

For  the  experimental  evaluation  of  CAI  training,  students  from  the  BE/E 
School  were  randomly  selected  and  received  a  segment  of  their  training 
by  means  of  CAI.  They  then  returned  to  class  for  the  remainder  of  their 
training.  A  total  of  760  students  were  used  during  student  tryouts  and 
operational  tests.  These  students  used  more  than  10,197  terminal  hours 
of  CAI. 


Findings  and  Conclusions 

Computer  assisted  instruction  provided  a  very  effective  method  of  train¬ 
ing.  CAI  students  scored  higher  than  class  instructed  students  on  both 
the  School  Examinations  and  the  Supplemental  Tests.  CAI  required  39% 
to  5W  less  training  time  than  class  instruction.  Students  gave  high 
ratings  to  their  CAI  instruction  and  would  prefer  to  have  about  70%  to 
Q0%  of  their  instruction  via  CAI. 


Extended  experience  with  CAI  did  not  have  a  detrimental  effect.  On 
the  contrary,  long  term  CAI  was  found  to  produce  a  small  improvement 
in  training  time  and  an  increase  in  the  students’  preference  for  CAI. 
Posttest  performance  was  not  affected. 

Computer  based  training  methods  are  ideally  suited  to  the  development 
of  good  instruction  because  the  computer  performance  recording  and  pro¬ 
cessing  capabilities  make  it  possible  to  improve  instruction  to  any 
required  level.  Data  analysis  and  summary  programs  were  developed  in 
this  project  which  allowed  lesson  designers  to  identify  specific  course 
weaknesses.  Course  revisions  based  on  these  analyses  resulted  in  marked 
improvements  in  instruction. 

Although  CAI  showed  about  a  k$l  time  savings  over  group  instruction, 
there  is  reason  to  believe  that  the  efficiency  of  CAI  can  be  improved 
even  further.  To  do  this,  increased  use  must  be  made  of  branching 
technology  in  order  to  provide  instruction  which  is  maximally  adaptive 
to  individual  differences.  A  number  of  advances  in  branching  technology 
were  made  in  this  project  and  incorporated  into  the  later  lessons.  The 
result  was  an  improvement  in  student  performance  over  the  previous  less 
adaptable  CAI.  In  addition,  it  was  found  that  student  option  was  just 
as  effective  as  control  of  training  by  means  of  pretests.  The  use  of 
student  option  simplifies  course  preparation.  Finally,  it  was  found 
that  dual  control  of  drill  and  practice  using  a  combination  of  program 
control  and  student  option  was  superior  to  strict  program  control. 

The  most  costly  and  time  consuming  part  of  CAI  course  development  was 
the  initial  preparation  of  basic  instruction  —  computer  coding  and 
programming  of  these  materials  took  far  less  effort.  CAI  shares  this 
materials  development  problem  with  all  forms  of  individualized  instruc¬ 
tion. 

The  evaluation  of  CAI  using  the  IBM  1500  system  was  made  in  terms  of 
technical  and  economic  feasibility.  These  included  such  factors  as 
training  effectiveness,  software  and  hardware  limitations,  operational 
limitations  and  cost  data.  CAI  using  the  1500  system  was  found  to  be 
highly  effective.  Serious  hardware  and  software  limitations  occurred 
in  programming  of  advanced  instructional  logics  in  COURSEWRITER  II, 
using  graphic  displays,  storing  sufficient  lesson  material  on-line, 
obtaining  short  system  response  times  with  complex  instructional  mate¬ 
rials,  and  using  the  random  access  audio.  A  major  operational  limita¬ 
tion  is  the  maximum  of  32  terminals  for  a  single  1500  system.  Capital 
costs  for  the  system  are  several  times  higher  on  a  per  terminal  basis 
than  newer  CAI  systems  with  larger  terminal  capacities. 

Systems  such  as  the  IBM  1500  system  can  compete  costwise  with  classroom 
instruction  in  some  applications.  However  they  are  generally  not  cost 
effective  in  comparison  to  individualized  instruction.  The  newer  CAI 
systems,  such  as  TICCET  and  PLATO  IV,  will  be  cost  effective  compared 
to  off-line  individualized  instruction,  and  will  provide  greatly  im¬ 
proved  computer  based  training  capabilities. 
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SUMMARY 


Problem 


This  project  is  part  of  Advanced.  Development  Objective  P43-03X,  Educa¬ 
tion  and  Training.  The  objectives  of  the  project  are  to  develop  com¬ 
puter  assisted  instruction  (CAl)  techniques  and  methods,  to  test  CAI 
training  modules  derived  from  these  techniques  and  to  investigate  the 
feasibility  of  integrating  CAI  into  Navy  training.  This  is  the  final 
report  on  the  development  and  evaluation  of  CAI  techniques  which  were 
derived  for  a  small  dedicated  computer  system.  The  computer  system 
used  was  the  IBM  1500  Instructional  System  and  the  curriculum  was  taken 
from  Basic  Electricity/Electronics  (BE/E)  School.  The  project  included 
three  major  development  and  evaluation  areas:  (l)  instructional  strat¬ 
egies  for  CAI,  (2)  computer  based  evaluation  and  revision  methods,  and 
(3)  empirical  evaluation  of  the  effectiveness  of  CAI. 


Approach 

The  development  of  CAI  lessons  consisted  of  a  series  of  steps  beginning 
with  identification  of  training  objectives  and  construction  of  criterion 
tests  and  continuing  with  planning  of  instructional  sequences  and  strat¬ 
egies,  preparation  of  lesson  materials,  review  and  revision  of  lessons, 
coding  and  checkout,  student  tryouts,  analysis  of  lesson  deficiencies, 
revision  of  lessons,  and  operational  tests.  In  the  process  of  develop¬ 
ing  CAI  materials  a  number  of  special  branching  strategies  were  devel¬ 
oped  to  provide  pretest  branching,  branched  remediation,  student  control 
of  instructional  sequences,  dual  control  of  practice  by  both  student  and 
program,  and  optional  reviews. 

The  analysis  of  student  performance  to  identify  lesson  deficiencies 
required  the  development  of  a  new  student  performance  data  management 
system.  A  complete  revision  methodology  was  also  developed  to  assist  in 
the  revision  process. 

For  the  experimental  evaluation  of  CAI  training,  students  from  the  BE/E 
School  were  randomly  selected  and  received  a  segment  of  their  training 
by  means  of  CAI.  They  then  returned  to  class  for  the  remainder  of  their 
training.  A  total  of  T60  students  were  used  during  student  tryouts  and 
operational  tests.  These  students  used  more  than  10,197  terminal  hours 
of  CAI. 


Findings  and  Conclusions 

Computer  assisted  instruction  provided  a  very  effective  method  of  train¬ 
ing.  CAI  students  scored  higher  than  class  instructed  students  on  both 
the  School  Examinations  and  the  Supplemental  Tests.  CAI  required  39% 
to  5 less  training  time  than  class  instruction.  Students  gave  high 
ratings  to  their  CAI  instruction  and  would  prefer  to  have  about  70$  to 
80$  of  their  instruction  via  CAI. 


iii 


r 


Extended  experience  with  CAI  did  not  have  a  detrimental  effect.  On 
the  contrary,  long  term  CAI  was  found  to  produce  a  small  improvement 
in  training  time  and  an  increase  in  the  students'  preference  for  CAI. 
Posttest  performance  was  not  affected. 

Computer  based  training  methods  are  ideally  suited  to  the  development 
of  good  instruction  because  the  computer  performance  recording  and  pro¬ 
cessing  capabilities  make  it  possible  to  improve  instruction  to  any 
required  level.  Data  analysis  and  summary  programs  were  developed  in 
this  project  which  allowed  lesson  designers  to  identify  specific  course 
weaknesses.  Course  revisions  based  on  these  analyses  resulted  in  marked 
improvements  in  instruction. 

Although  CAI  showed  about  a  b^%  time  savings  over  group  instruction, 
there  is  reason  to  believe  that  the  efficiency  of  CAI  can  be  improved 
even  further.  To  do  this,  increased  use  must  be  made  of  branching 
technology  in  order  to  provide  instruction  which  is  maximally  adaptive 
to  individual  differences.  A  number  of  advances  in  branching  technology 
were  made  in  this  project  and  incorporated  into  the  later  lessons.  The 
result  was  an  improvement  in  student  performance  over  the  previous  less 
adaptable  CAI.  In  addition,  it  was  found  that  student  option  was  just 
as  effective  as  control  of  training  by  means  of  pretests.  The  use  of 
student  option  simplifies  course  preparation.  Finally,  it  was  found 
that  dual  control  of  drill  and  practice  using  a  combination  of  program 
control  and  student  option  was  superior  to  strict  program  control. 

The  most  costly  and  time  consuming  part  of  CAI  course  development  was 
the  initial  preparation  of  basic  instruction  —  computer  coding  and 
programming  of  these  materials  took  far  less  effort.  CAI  shares  this 
materials  development  problem  with  all  forms  of  individualized  instruc¬ 
tion. 

The  evaluation  of  CAI  using  the  IBM  1500  system  was  made  in  terms  of 
technical  and  economic  feasibility.  These  included  such  factors  as 
training  effectiveness,  software  and  hardware  limitations,  operational 
limitations  and  cost  data.  CAI  using  the  1500  system  was  found  to  be 
highly  effective.  Serious  hardware  and  software  limitations  occurred 
in  programming  of  advanced  instructional  logics  in  COURSEWRITEB  II, 
using  graphic  displays,  storing  sufficient  lesson  material  on-line, 
obtaining  short  system  response  times  with  complex  instructional  mate¬ 
rials,  and  using  the  random  access  audio.  A  major  operational  limita¬ 
tion  is  the  maximum  of  32  terminals  for  a  single  1500  system.  Capital 
costs  for  the  system  are  several  times  higher  on  a  per  terminal  basis 
than  newer  CAI  systems  with  larger  terminal  capacities. 

Systems  such  as  the  IBM  1500  system  can  compete  costwise  with  classroom 
instruction  in  some  applications.  However  they  are  generally  not  cost 
effective  in  comparison  to  individualized  instruction.  The  newer  CAI 
systems,  such  as  TICCET  and  PLATO  IV,  will  be  cost  effective  compared 
to  off-line  individualized  instruction,  and  will  provide  greatly  im¬ 
proved  computer  based  training  capabilities. 
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COMPUTER  ASSISTED  INSTRUCTION  IN  NAVY  TECHNICAL  TRAINING 
USING  A  SMALL  DEDICATED  COMPUTER  SYSTEM:  FINAL  REPORT 


I.  Description  of  the  Project 


A.  Mission 


The  objectives  of  this  project  are  to  develop  computer  assisted  instruc¬ 
tion  (CAl)  techniques  and  methods,  to  test  CAI  training  modules  derived  from 
these  techniques  and  to  investigate  the  feasibility  of  integrating  CAI  into 
Navy  training.  This  is  the  final  report  on  the  development  and  evaluation 
of  CAI  techniques  which  were  derived  for  a  small  dedicated  computer  system. 
The  computer  system  used  was  the  IBM  1500  Instructional  System  and  the  cur¬ 
riculum  was  taken  from  Basic  Electricity/Electronics  (BE/E)  School. 

This  project  is  part  of  Advanced  Development  Objective  PU3-03X,  Educa¬ 
tion  and  Training.  Other  projects  in  that  ADO  which  test  applications  of 
computers  to  instruction  include  the  CAI  project  at  the  Naval  Academy 
(Final  Report,  1971)  and  the  computer  managed  instruction  (CMl)  project  at 
the  Naval  Technical  Training  Command,  Memphis. 

The  objective  of  ADO  P43-03X  is  to  test  the  feasibility  of  available 
new  advances  in  training  technology,  performance  standards,  proficiency 
measurement,  and  training  equipment  as  a  means  for  providing  and  maintain¬ 
ing  increased  personnel  capability.  The  Navy  faces  increasing  demands  for 
highly  skilled  personnel  to  operate  and  maintain  complex  electronic  equip¬ 
ment.  There  is  a  need  to  provide  training  which  is  more  job  related,  can 
be  accomplished  in  a  shorter  period  of  time.,  and  is  adaptive  to  a  wide 
range  of  backgrounds  and  aptitudes  in  the  recruit  population. 

Much  of  Navy  training  uses  classroom  instruction.  There  are  inherent 
limitations  in  this  type  of  training,  including  the  following:  (l)  in¬ 
adequate  mastery  of  all  of  the  training  objectives  by  some  of  the  students, 
(2)  waste  of  student  time  due  to  inefficient  learning  sequences  and  pacing 
of  class  to  average  or  below  average  learning  rate,  (3)  variation  in  quality 
of  instruction  both  across  instructors  and  by  the  same  instructor  from  day- 
to-day,  and  (k)  patterning  instruction  to  some  hypothetical  average  rather 
than  to  the  needs  of  individuals  in  rate  and  type  of  learning  required  to 
achieve  training  objectives.  Conventional  classroom  methods  critically 
limit,  or  render  nearly  impossible,  efforts  to  individualize  instruction. 
Furthermore,  classroom  effectiveness  is  subject  to  variation  because  of 
instructor  differences  and  stressful  conditions  such  as  time  pressure  and 
large  class  size.  A  high  turnover  rate  among  instructors  aggravates  these 
problems  by  constantly  eroding  experience,  instructional  competency,  and 
knowledge  of  local  classroom  innovations.  Without  drastic  changes  in  the 
technology  of  classroom  operation,  it  seems  highly  unlikely  that  little 
more  than  minor  improvement  can  be  expected. 
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The  application  of  computer  technology  to  the  individualizing  of  instruc¬ 
tion  provides  a  promising  means  of  overcoming  the  deficiencies  of  classroom 
instruction.  Computer  assisted  instruction  permits:  (l)  precise  control  of 
presentation  of  information  to  the  student,  (2)  immediate  processing  of 
student  responses  combined  with  presentation  of  appropriate  "feedback" 
information,  (3)  branching  or  altering  of  sequence  of  training  contingent 
upon  student’s  progress,  and  (1+)  collection  and  processing  of  highly  detailed 
student  response  data. 

B.  CAI  Problems  Investigated 

As  the  project  progressed  three  major  development  areas  became  defined 
more  precisely:  (l)  instructional  strategies  for  CAI,  (2)  computer  based 
evaluation  and  revision  methods,  and  (3)  development,  evaluation  and  revision 
of  CAI  course  materials  for  the  target  application.  A  general  evaluation  of 
CAI  was  made  in  two  ways.  First,  a  comparison  was  made  of  CAI  with  standard 
classroom  instruction.  These  results  are  described  in  the  first  two  parts 
of  Section  III.  Second,  and  more  important  from  a  research  and  development 
standpoint,  was  the  development  and  evaluation  of  new  CAI  technologies. 

These  new  technologies  are  described  in  Section  II  and  results  of  evaluation 
are  described  in  Section  III.  An  evaluation  of  the  1500  system  is  given  in 
Section  IV.  Such  an  evaluation  must  include  the  entire  matrix  of  instruc¬ 
tional  technologies  developed  for  the  system  and  not  solely  the  technical 
or  engineering  characteristics  of  the  computer  and  terminals.  Thus,  other 
sections  of  the  report  also  relate  to  the  evaluation  of  CAI  which  employ 
the  1500  system  as  the  materials  development  and  delivery  system. 

C .  Staff  Resources  and  Organization 

To  conduct  a  CAI  project  a  mix  of  staff  skills  and  resources  is  needed. 

One  set  of  resources  involves  skills  in  research  methods,  research  design, 
conducting  experiments,  analyzing  and  interpreting  results,  and  developing 
new  instructional  techniques  and  strategies.  Staff  persons  having  these 
skills  were  research  psychologists  and  education  specialists.  At  maximum, 
there  were  ten  persons  on  the  project  having  these  skills  but  by  the  con¬ 
clusion  of  the  1500  evaluation  project  there  were  eight,  because  not  all 
vacancies  created  by  turnover  were  filled.  A  second  set  of  skills  were 
those  related  to  computer  programming  and  operation.  System  and  applica¬ 
tion  programming  as  well  as  computer  operation  and  computer  aid  skills  were 
represented  in  this  group.  Early  in  the  project  there  were  four  persons  in 
this  group  but  later  the  number  was  reduced  to  three.  Subject  matter  skills 
were  provided  by  two  military  instructors  assigned  from  the  Basic  Electricity/ 
Electronics  School,  Service  School  Command,  Naval  Training  Center,  San  Diego. 
Later  this  number  was  reduced  to  one  billet.  Finally,  on-line  coding  and 
debugging  of  lessons,  which  had  been  designed  by  the  professional  and 
technical  staff,  was  provided  by  four  part  time  California  State  University 
students.  They  also  assisted  in  the  tryout  and  evaluation  of  CAI  materials. 
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The  staff  was  organized  into  two  CAI  design  and  evaluation  groups,  each 
under  the  direction  of  a  Program  Director.  Each  group  was  responsible  for 
the  entire  cycle  of  design  and  experimental  evaluation  of  CAI  course  modules 
including  the  development  of  experimental  CAI  techniques.  The  two  groups 
operated  in  parallel  and  were  in  different  stages  of  the  design  process  at 
any  given  time.  This  prevented  peak  loads  or  demands  from  building  up  on 
limited  resources,  such  as  terminal  capacity,  student  performance  data 
processing  using  the  1500  system,  and  the  availability  of  students  from  the 
school . 

D.  1500  System  and  Instructional  Facilities 

The  IBM  1500  Computer  System  was  installed  April  1968  and  remained 
operational  continuously  until  removed  on  30  June  1972 •  Computer  hardware 
as  first  installed  consisted  of  an  IBM  1130  Computer  with  32K  memory  and 
the  following  on-line  equipment:  a  line  printer  (80  lines  per  minute)  and 
five  disk  drives  with  removable  single  cartridge  disks.  A  multiplexing 
station  control  unit  was  the  connecting  link  to  the  eight  terminal  stations. 
The  terminals  were  equipped  with  both  an  image  projector  and  a  cathode  ray 
tube  (CRT)  display  unit  with  keyboard  and  light  pen  (Figure  l).  The  image 
projector  utilized  a  film  cartridge  that  allowed  random  access  to  1000 
images.  In  addition,  three  of  the  terminals  had  typewriters  which  could 
produce  hard  copy  for  special  requirements.  In  1969  the  system  was  enlarged 
to  l6  terminals  and  two  tape  drives  were  added  to  the  system.  Random  access 
audio  units  were  installed  at  12  of  the  terminals. 

The  equipment  was  installed  in  two  converted  classrooms  and  a  small 
storage  room.  Air  conditioning  was  installed  to  provide  the  environment 
necessary  for  the  computer  hardware.  The  IBM  1130  computer  and  on-line 
peripheral  equipment  plus  two  terminal  stations  occupied  one  room.  The  ih 
remaining  terminal  stations  were  assigned  to  two  rooms  (see  Figure  l).  Each 
student  carrel  was  designed  to  allow  room  for  both  computer  controlled  equip¬ 
ment  and  auxiliary  equipment.  Auxiliary  laboratory  equipment  consisted  of 
audio  signal  generators,  simple  oscilloscopes,  vaccuum  tube  voltmeters, 
multimeters,  NEAT  boards  (Navy  Electronics  Application  Trainers),  capacitor 
displays,  and  lamp  boards.  Auxiliary  laboratory  equipment,  except  for  the 
capacitor  displays  and  lamp  boards,  was  furnished  by  the  Basic  Electricity/ 
Electronics  School  and  was  the  same  type  used  by  the  school.  In  some  lessons, 
the  auxiliary  equipment  was  used  by  students  under  guidance  of  the  computer. 
The  computer  provided  help  options  and  evaluated  student  measurements. 

The  1500  operating  system  included  an  executive  program  for  both  time 
sharing  and  data  processing  modes.  The  1130  system  monitor  also  provided 
additional  data  processing  capabilities.  A  number  of  utilities  were  supplied 
with  the  system,  such  as  disk-to-card,  card-to-disk,  tape-to-disk,  and  disk- 
to-tape.  Data  processing  programs  were  not  supplied  by  IBM  but  were  developed 
as  part  of  this  project.  These  included  a  student  performance  data  extraction 
program  and  several  sort  and  data  summary  programs  to  provide  student  perfor¬ 
mance  data  to  authors  for  lesson  evaluation  and  revision. 
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Figure  1.  Top:  Student  Room  Layout.  Bottom:  Carrel  and  1500  Equipment 
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E.  Basic  Electricity/Electronics  School 


The  curriculum  of  the  Basic  Electricity/Electronics  School  represents 
a  common  core  of  knowledges  and  skills  in  the  fundamentals  of  electricity 
and  electronics  as  needed  for  subsequent  training  in  a  number  of  Navy  ratings. 
Thus,  the  school  is  a  feeder  school  for  "A"  schools  for  a  variety  of  ratings: 
Aviation  Electrician* s  Mate,  Aviation  Electronics  Technician,  Communications 
Technician,  Data  Processing  Technician,  Data  Systems  Technician,  Electrician* s 
Mate,  Electronics  Technician,  Fire  Control  Technician,  Interior  Communications 
Electrician,  Sonar  Technician  (Surface),  Sonar  Technician  (Submarines), 
Torpedoman,  and  Electronics  Warfare  Technician. 

A  large  percentage  of  the  students  enter  the  school  following  recruit 
training.  The  remaining  students  are  assigned  from  the  fleet.  The  recruits 
selected  are  middle  to  high  aptitude  individuals  as  measured  by  classifica¬ 
tion  tests. 

The  selection  criteria  vary  for  the  ’’A**  schools.  Different  combinations 
of  two  or  three  aptitude  tests  are  used  by  the  schools  and  the  minimum 
entrance  scores  vary.  In  addition,  interviewers  for  recruits,  and  commanding 
officers  for  students  from  the  fleet,  can  recommend  exceptions  to  the  quali¬ 
fication  requirements.  There  is  wide  variation  also  in  previous  background 
in  or  knowledge  of  electronics.  The  range  is  from  no  high  school  related 
experience  to  two  or  more  years  of  college. 

The  student  load  fluctuates  during  the  year.  Student  aptitude  varies 
on  a  seasonal  basis,  probably  reflecting  school  graduation  schedules.  The 
on  board  count  averages  300  to  500  students. 

Originally,  the  school  had  three  tracks  of  seven,  eight,  and  nine  weeks, 
with  the  lower  aptitude  students  on  the  longer  tracks.  In  1968  the  tracks 
were  shortened  to  six,  seven,  and  eight  weeks.  At  the  time  the  CAI  AC  Module 
was  tested,  there  were  seven  or  eight  new  classes  convening  each  week. 

Students  were  divided  into  four  different  ability  tracks  and  the  time  to 
complete  the  class  instructed  course  was  six  to  eight  weeks,  depending  on 
the  track. 

In  1969  all  students  were  put  on  a  single  six-week  class  instructed 
track.  This  arrangement  existed  during  the  operational  testing  of  the  CAI 
Inductance  Module,  Capacitance  Module,  DC  Module  and  the  AC  Series  Circuits 
and  Resonance  Module.  The  last  classes  of  six-week  track  students,  before 
full  operation  of  the  new  individualized  training  curriculum,  were  used  in 
the  Long  Term  CAI  Study. 

The  School  curriculum  during  the  above  period  was  divided  into  eight 
topic  areas  of  instruction  given  over  a  six  week  period.  Class  instruction 
consisted  primarily  of  lectures,  question  and  answer  discussion  periods, 
drill  and  practice  sessions,  assignment  sheets,  and  demonstrations  and  short 
laboratory  exercises.  Course  credit  for  each  area  was  determined  by  an 
Area  Examination.  A  summary  outline  of  each  area  and  the  scheduled  train¬ 
ing  time  is  shown  below: 
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BE/E  School  Curriculum 


Allotted  Class  Time 


Area 

i 

Matter /Electrostatic s/Magnet ism 

12  hours 

Area 

2 

Voltage /Current /Resistance 

20  hours 

Area 

3 

DC  Circuits 

3^  hours 

Area 

U 

AC  Theory 

lb  hours 

Area 

5 

Inductance /Capacitance 

3b  hours 

Area 

6 

AC  Series  Circuits 

3b  hours 

Area 

7 

AC  Parallel  Circuits 

3^  hours 

Area 

8 

Power  Transformers 

8  hours 

During  1970  and  1971  the  classroom  curriculum  was  replaced  with  an 
individualized  learning  program,  called  Basic  Electricity/Electronics 
Individualized  Learning  System  (BEEINLES).  There  were  enough  changes  in 
the  objectives,  and  hence  the  content  of  training,  such  that  the  classroom 
and  individualized  curricula  were  not  compatible  or  interchangeable. 

Separate  tests  were  used  for  the  two  programs.  The  BEEINLES  curriculum  was 
divided  into  15  units  called  modules.  Each  module  consisted  of  a  Module 
Booklet  divided  into  self  instructional  lessons.  Each  lesson  contained  a 
list  of  training  objectives,  an  overview,  a  narrative,  summary,  programmed 
instructional  materials,  and  a  self  administered  progress  test.  Supplemental 
materials  included  audio  instruction  via  cassette  and  film  strip  cartridges. 
Each  student  was  free  to  use  any  part  or  all  of  the  available  materials  and 
to  choose  when  he  would  take  the  Module  Examination  which  required  a  passing 
score  before  continuing.  This  change  in  curriculum  and  training  objectives 
made  it  necessary  to  revise  the  CAI  AC  Series  Circuits  and  Resonance  Module 
into  a  new  module,  CAI  AC  Series  Circuits,  for  the  experimental  evaluation 
of  student  option  versus  program  controlled  branching  methods. 


II.  Development  of  CAI  Technology 
A.  CAI  Development  and  Revision  Process 
1.  Module  Development 

The  steps  in  design  and  evaluation  of  each  CAI  module  are  flowcharted 
in  Figure  2.  Development  of  each  CAI  module  began  by  using  a  team  approach 
to  determine  and  specify  the  course  training  objectives.  The  CAI  training 
objectives  were  locked  into  the  training  objectives  of  the  Basic  Electricity/ 
Electronics  School  curriculum.  This  was  necessary  because  the  students  who 
were  used  in  this  project  were  taken  from  regular  classes  and  had  to  be 
trained  on  the  same  training  objectives  as  all  other  students. 

There  were  three  major  sources  for  the  training  objectives:  (l) 
Preliminary  Instructor’s  Guide  (August  1967),  (2)  Trainee’s  Textbook, 
Fundamentals  of  Electronics,  (NAVPERS  93^00A-lb),  and  (3)  Trainee’s  Guide, 
(NAVPERS  935l8-2b) .  In  addition,  the  School  Area  Examinations  were  checked 
to  cross  reference  all  objectives  actually  tested.  Because  most  of  the 
school  objectives  were  stated  in  general  terms,  it  was  necessary  to  rewrite 
them  into  behavioral  statements  which  specifically  defined  the  desired 
terminal  criterion  behavior  of  the  student.  A  behaviorally  stated  CAI 
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training  objective  specified  the  exact  stimulus  conditions  to  which  the 
student  would  respond,  the  required  response  and  sometimes  the  level  of 
performance  required.  The  purpose  of  behavioral  CAI  training  objectives 
was  to  provide  CAI  authors  with  a  clear  statement  of  the  material  to  be 
taught  and  to  assist  in  the  development  of  criterion  questions  to  evaluate 
CAI  training.  This  also  allowed  the  authors  to  select  appropriate  train¬ 
ing  strategies  and  to  save  training  time  by  eliminating  irrelevant  instruc¬ 
tion. 


Figure  2.  Steps  in  design  and  evaluation  of 
CAI  course  module. 


After  all  behaviorally  stated  training  objectives  were  completed  the 
second  step  consisted  of  grouping  them  into  meaningful  content  units.  Each 
content  unit  was  then  designated  to  become  one  or  more  individual  CAI 
lessons.  Finally  the  CAI  lessons  were  assigned  into  a  tentative  sequence 
within  the  module.  The  school  curriculum  prescribed  a  sequence  of  training 
objectives  and  lesson  plans.  Because  of  the  way  students  were  shunted  to 
CAI  modules  for  training  and  then  returned  to  the  class  for  the  remainder 
of  their  training,  the  overall  sequence  of  topics  or  areas  in  the  school 
curriculum  had  to  be  followed.  However,  sequencing  within  each  area  was 
under  control  of  lesson  designers  who  planned  and  wrote  the  CAI  lessons.  In 
this  sequencing,  the  hierarchical  arrangement  of  concepts  within  the  subject 
matter  was  emphasized.  Because  of  the  progressive  building  of  concepts  and 
principles  in  electronics,  attention  was  given  to  sequencing  which  would 
facilitate  transfer  to  the  learning  of  higher  order  concepts  and  principles. 
The  initial  sequencing  which  was  decided  upon  was  subject  to  evaluation  and 
revision  based  upon  empirical  data  from  student  tryouts. 


T 


2 .  Lesson  Development 

Individual  authors  of  the  team  were  responsible  for  detailed  develop¬ 
ment  of  their  assigned  lessons.  After  lessons  were  assigned  to  individual 
authors,  steps  in  the  development  of  the  lessons  consisted  of:  (l)  background 
familiarization  with  subject  matter  and  determining  instructional  sequence 
and  strategy;  (2)  writing  criterion  questions  for  each  training  objective; 

(3)  writing  the  instructional  sequences  for  each  training  objective;  (k) 
team  leader  and  subject  matter  specialist’s  critical  review  of  drafted  CAI 
lessons  followed  by  corresponding  corrections  and  changes  by  the  author; 

(5)  on-line  coding  of  course  materials  and  author  checkout  and  debugging; 

(6)  module  tryouts  for  revision  evaluation;  and  (7)  revisions. 

a.  Criterion  test  items.  An  integral  part  of  developing  CAI 
lessons  was  the  writing  of  criterion  test  items  to  be  used  to  measure  and 
evaluate  training  effectiveness.  The  actual  writing  of  criterion  test 
items  also  served  to  emphasize  to  authors  the  desired  terminal  behavior 
required  by  the  training  objectives. 

A  set  of  parallel  criterion  questions  was  written  for  each  behavioral 
objective  and  used  at  three  specific  points  during  and  after  training: 

(1)  Immediately  after  a  sequence  of  training  frames  at  a  point 
where  the  author  could  adequately  determine  the  effectiveness  of  the  train¬ 
ing  strategy  and  often  at  the  end  of  the  lesson  as  part  of  a  drill  and 
practice  review; 

(2)  Within  the  Lesson  Test  to  evaluate  short-term  memory  and 
retention;  and 

(3)  Within  the  Supplemental  Test  at  the  completion  of  the 
module  to  measure  long-term  memory. 

There  were  usually  four  forms  of  each  School  Area  Examination.  Item 
analyses  of  these  forms  showed  that  U6  to  6h%  of  the  CAI  behavioral  training 
objectives  were  either  untested  or  tested  on  less  than  two  forms  of  an 
Examination.  The  School  Examinations  were,  therefore,  "norm-referenced"  and 
designed  to  evaluate  a  student’s  performance  relative  to  other  students  and 
not  to  the  training  criteria.  For  this  reason  Supplemental  Tests  were 
developed  to  evaluate  performance  on  the  "under  tested"  training  objectives. 
Thus  together,  any  form  of  the  School  Examination  plus  the  corresponding 
Supplemental  Test  evaluated  approximately  95$  of  the  total  training  objec¬ 
tives  and  provided  a  "criterion-referenced"  measurement  for  comparing  train¬ 
ing  methods . 

b.  Instructional  sequencing  and  strategies.  Derivation  of  training 
objectives  and  criterion  test  items  focused  upon  the  content  of  behaviors 
and  skills  to  be  learned.  Sequencing  of  objectives  required  consideration 

of  both  the  content  organization  inherent  in  the  subject  matter  and  the  types 
of  learning  which  were  required  by  the  objectives.  Instructional  strategies 
focused  upon  the  types  of  learning  required  by  the  training  objectives,  the 
conditions  of  learning,  and  types  of  learning  diff iculties  encountered  by 
the  population  of  students  who  receive  the  training. 
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Individual  authors  were  allowed  to  develop  individual  lesson 
styles  during  the  first  generation  CAI  modules,  AC  Theory  and  Inductance. 

These  modules  were  developed  concurrently  by  two  different  CAI  course  develop¬ 
ment  teams.  The  resulting  CAI  course  materials  followed  a  fixed  sequence 
instructional  design  which  required  all  students  to  receive  training  across 
all  training  objectives.  Individualization  of  instruction  was  limited  to 
giving  differential  response  feedback,  within  frame  branching,  some  back 
branching  within  training  and  practice  sequences,  and  optional  student 
branching  review.  In  other  words,  except  for  variations  in  the  instruction 
just  described,  all  students  were  given  fixed  sequence  training  on  all 
training  objectives  in  every  CAI  lesson. 

At  the  onset  of  planning  the  second  generation  CAI  modules, 
Capacitance,  Direct  Current,  and  Series  Circuits  and  Resonance,  the  authoring 
teams  began  examining  the  possibilities  of  making  greater  usage  of  computer 
capabilities  to  adapt  CAI  to  meet  the  training  needs  of  individual  students. 
Although  fixed  sequence  CAI  proved  to  be  superior  to  classroom  instruction 
in  previous  modules,  it  still  retained  the  inherent  disadvantage  of  provid¬ 
ing  either  too  little  training  for  the  slow  learning  or  low  aptitude  student 
or  too  much  training  for  the  sophisticated,  fast  learning,  or  high  aptitude 
student.  It  was  concluded  that  the  lesson  programing  logics  controlling  the 
flow  of  instruction  would  have  to  be  highly  sensitive  to  the  response 
history  and  learning  characteristics  of  each  student  in  order  to  achieve  the 
goal  of  adaptive  CAI  training.  A  major  approach  to  this  problem  was  the 
use  of  branching  logics  which  would  allow  either  the  program  or  the  student 
to  determine  the  amount  of  instruction  and/or  practice  received  on  each 
behavioral  training  objective.  These  strategies  included  pretest  branching, 
student  controlled  training,  optional  practice,  dual-control  practice,  and 
optional  review.  Section  II. B.  discusses  details  of  these  instructional 
strategies . 


c.  Lesson  material  preparation.  Lesson  preparation  is  the  most 
difficult  of  all  the  steps  in  CAI  course  development.  There  is  a  strong 
tendency  for  lesson  designers  who  have  had  teaching  experience  to  write 
"lecture  frames."  Only  after  these  habits  are  unlearned  do  lesson  designers 
become  effective  in  writing  instructional  CAI  materials. 

The  preparation  of  lesson  material  included  the  writing  of  the 
actual  training  content,  determining  when  and  what  kinds  of  responses  were 
to  be  required  of  students,  specifying  the  conditions  of  branching  and 
student  data  to  be  used  for  making  branching  decisions,  preparing  drafts  of 
all  visual  presentations  and  where  appropriate  writing  audio  scripts  to  be 
used  in  conjunction  with  the  lessons.  Authors  made  extensive  use  of  flow 
charts  as  an  aid  whenever  complex  branching  logics  were  employed  in  a  lesson. 

When  all  training  sequences,  optional  reviews  and  special  branch¬ 
ing  sequences  were  written,  the  next  step  for  the  author  was  to  transpose 
each  frame  onto  a  special  frame-planning  form.  In  summary,  the  "Author- 
Coder  Frame  Planning  Guide"  was  a  form,  developed  within  this  project,  which 
enabled  lesson  designers  to  plan  the  layout  of  each  frame  of  a  lesson.  In 
addition  to  the  instructional  content  and  visual  stimulus  format,  all  control 
information  necessary  for  computer  programing  was  also  entered  on  this  form. 
This  method  greatly  facilitated  the  preparation  and  editing  of  lesson 
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material.  It  was  also  an  efficient  way  to  document  and  transmit  the  lesson 
to  the  person  responsible  for  the  next  step  of  coding  and  program  checkout. 

d.  Author-coder  frame  planning  guide.  An  example  of  use  of  the 
frame  planning  guide  is  shown  in  Figure  3.  Notations  on  the  frame  planning 

AUTHOR-CODER  FRAME  PLANNING  GUIDE 
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Figure  3.  Example  of  Frame  Planning  Guide 
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form  indicated  the  types  of  control  and  flow  of  the  program  which  the 
author  wished  implemented.  Detailed  explanation  of  coding  used  for  the 
Frame  Identifier  and  Response  Type  are  given  in  Section  II.  C. ,  Data 
Management  System.  In  summary,  the  10  digit  Frame  Identifier  Code  served  to: 
(l)  uniquely  identify  each  frame  in  the  lesson,  (2)  identify  frame  as  to 
lesson  and  training  objective,  (3)  classify  the  type  of  frame  ( instruct ionally) 
and  identify  the  consequence  of  answering  incorrectly,  and  (4)  provide  for 
special  author  code  use.  The  two  digit  Response  Type  Identifier  served  to: 

(l)  indicate  the  exact  answer  selected,  (2)  identify  the  correctness  of  the 
answer,  and  (3)  specify  the  exact  student  feedback  for  selection  of  par¬ 
ticular  answers.  Other  information  presented  on  the  form  included  the 
actual  text  and  its  format  on  the  screen  as  seen  by  the  student,  whether  or 
not  an  image  would  be  shown  on  the  projector,  the  code  name  of  the  image, 
and  branching  instructions  as  a  consequence  of  student  responses. 

e.  Team  leader  and  subject  matter  specialist^  review.  It  was  a 
standard  procedure  for  the  team  leader  to  review  each  author fs  lesson 
materials  for  pedagogical  adequacy  and  adherance  to  project  conventions  of 
format,  response  made,  and  identification  coding.  A  subject  matter  special¬ 
ist  also  reviewed  the  material  for  technical  correctness.  Before  turning 
the  lesson  over  to  coders,  the  author  made  appropriate  revisions  to  the 
lesson  materials. 

3.  Coding  and  Programming 

Once  the  CAI  lessons  were  designed  they  were  coded  in  COURSEWRITER  II, 
an  authoring  language  developed  by  IBM.  Initially,  this  was  done  by  prepar¬ 
ing  coding  sheets  from  which  keypunched  lesson  decks  were  prepared  and  read 
into  computer  storage.  This  method  proved  to  be  both  time  consuming  and 
error  prone.  Also,  updating  of  lesson  decks  as  lessons  were  modified  became 
a  major  problem. 

Early  in  the  project  a  transition  was  made  to  direct  on-line  coding 
instead  of  punched  card  input.  Students  from  California  State  University, 

San  Diego,  who  worked  on  a  part  time  basis,  were  trained  to  code  on-line 
at  terminals  using  the  lesson  planning  guides.  Once  coded,  the  lesson 
material  could  be  executed  in  the  same  manner  in  which  a  student  would  see 
the  lesson.  This  capability  permitted  prompt,  on-line  debugging  of  lessons. 

It  is  estimated  that  the  shift  to  on-line  coding  and  debugging  reduced  the 
time  of  the  coding  step  by  at  least  30  percent. 

Instructional  requirements  often  revealed  the  need  for  capabilities 
not  available  in  COURSEWRITER .  The  language  had  a  provision  which  permitted 
the  writing  of  special  functions  which  could  be  added  to  the  lesson  code. 

These  functions  added  capabilities  for  answer  analysis,  special  lesson 
controls  and  data  manipulation. 

4 .  Film  and  Audio  Preparation 

a.  Film  preparation.  Preparation  of  film  involved  the  following 
steps:  (l)  Authors  prepared  rough  sketches  of  required  illustrations;  (2) 

Laboratory  technical  illustrators  prepared  finished  art;  (3)  Art  was 
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photographed  with  a  16mm  camera  (Bolex  Hl6)  (averaged  eight  man  hours  of 
time/module);  (U)  Film  was  processed  and  examined  for  quality  and  correct 
matching  of  image  and  address  code  (one  day);  (5)  Copies  were  made  from 
master  (averaged  10  days);  (6)  Special  leaders  and  trailers  were  placed 
on  student  copies  and  mounted  into  special  cartridges  (one  day). 

Due  to  problems  in  camera  equipment  and  long  film  processing 
schedules,  film  preparation  often  delayed  tryout  and  operational  CAI  student 
testing.  The  camera,  in  unknown  mechanical  condition,  had  to  be  borrowed 
from  other  Navy  facilities  for  each  filming.  Preaddressed  coded  film  was 
unavailable  during  the  first  two  years  of  the  project;  raw  film  required  an 
extra  processing  step  of  two  weeks.  Preaddressed  film  demanded  accurate 
calibration  in  camera  during  filming.  Errors  in  filming,  addressing,  or 
calibration  were  undetectable  until  the  finished  master  was  available. 

Errors  could  only  be  corrected  by  refilming.  Preparation  times  from 
photographing  to  final  correct  copies  of  four  to  eight  weeks  were  not 
uncommon . 

b.  Audio  preparation.  The  random  access  audio  device  allowed 
authors  to  present  audio  instructions  and  directions  in  combination  with 
the  visual  displays:  the  CRT,  image  projector  and  off-line  equipment. 

Actual  preparation  of  audio  materials  required  three  major  steps 
preprocessing,  assembly,  and  postprocessing.  Preprocessing  included  writing 
actual  audio  messages,  making  a  narrative  tape,  and  a  master  addressed 
cartridge  containing  an  addressed  digital  track.  Assembly  involved  the 
process  of  transcribing  the  narrative  messages  onto  a  master  addressed 
cartridge.  Finally  postprocessing  consisted  of  substituting  the  actual 
location  of  each  message  into  the  COURSEWRITER  II  program.  Any  change  in 
the  audio  messages  or  addresses  required  repeating  the  entire  audio  prep¬ 
aration  process.  Student  cartridges  could  be  reproduced  from  the  master 
cartridge  for  operational  use. 

Aside  from  the  multiple-step  preparation  process  for  production 
and  revision  of  audio  materials,  difficulties  in  audio  tape  preparations 
were  minimal. 

B.  Instructional  Strategies 

1 .  Module  Organization  and  Sequencing 

In  all  modules,  three  principles  were  emphasized  in  organizing  the 
instruction  into  lessons.  (l)  Heirarchical  sequencing.  Principles  and 
information  considered  necessary  or  useful  for  later  learning  were  taught 
first.  (2)  Transfer.  Sequencing  was  designed  to  facilitate  transfer  to 
the  learning  of  higher  order  concepts  and  principles.  (3)  Self-pacing. 

CAI  students  were  allowed  to  work  completely  at  their  own  rate  of  speed  and 
could  take  breaks  whenever  they  wished.  In  order  to  provide  convenient 
points  for  lesson  tests  and  for  taking  breaks,  lessons  were  designed  to  be 
less  than  one  hour  in  length  for  the  typical  student. 
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Two  other  sequencing  principles  were  frequently  found  to  be  appro¬ 
priate.  (1)  Spacing  of  difficult  discriminations.  The  most  difficult 
parts  of  the  course  were  often  those  in  which  the  students  had  to  learn  to 
discriminate  between  a  number  of  complex,  highly  similar  concepts.  For 
instance  all  the  types  of  magnitude  measurements,  and  their  alternate  names 
and  uses,  were  traditionally  taught  at  the  same  time.  For  the  CAI  training 
on  AC  Theory,  only  part  of  these  concepts  were  introduced  in  the  early 
lessons.  Then  when  related  concepts  were  taught  in  later  lessons,  the  new 
concepts  could  be  contrasted  with  ones  the  student  already  knew.  The  aim 
of  this  procedure  was  to  reduce  the  amount  of  interference  among  the  concepts 
introduced  early.  (2)  Concrete  to  abstract  sequencing.  This  type  of 
sequencing  was  frequently  used  to  provide  a  useful  frame  of  reference  for 
the  student  for  learning  the  more  theoretical  parts  of  the  course.  Thus, 
"hands-on"  lab  work  occurred  in  the  first  lessons  of  both  the  DC  and  AC 
Theory  modules.  Alternator  uses  and  construction  were  taught  in  the  second 
lesson  of  AC  Theory.  Demonstration  of  the  time  constant  and  identification 
of  types  of  capacitors  were  introduced  as  early  as  possible  in  the  capaci¬ 
tance  module. 

2.  Lesson  Organization  and  Frame  Construction 

Most  of  the  sequencing  principles  discussed  above  were  also  applied 
to  each  lesson.  In  addition  each  lesson  was  organized  into  a  student  "self- 
evaluation  sequence"  as  follows: 

a.  Reads  brief  overview  (in  Study  Guide) 

b.  Does  part  A 

(1)  reads  objective(s)  for  part  A  (in  guide) 

(2)  does  training  frames 

(3)  takes  criterion  frame(s) 

c.  Does  parts  B,  C,  etc. 

d.  Reviews 

e.  Takes  lesson  test 

The  objectives  and  overview  were  in  a  Study  Guide  which  the  student  could 
keep  and  could  refer  to  at  any  time  except  when  taking  tests.  In  some 
lessons  the  training  could  be  skipped,  either  by  passing  a  pretest  or  by 
taking  advantage  of  student  options.  In  some  lessons  an  on-line  optional 
review  was  available  in  step  d.  The  student  could  review  any  of  the  topics 
covered  in  the  lesson.  This  review  always  consisted  of  criterion  type 
problems  in  which  the  student  received  no  help  unless  he  missed  the  question. 

The  CAI  training  generally  followed  a  tutorial  model  designed  to 
take  maximum  advantage  of  the  information  retrieval  and  response  processing 
capabilities  of  the  computer.  Almost  all  of  the  training  frames  required 
an  active  response  on  the  part  of  the  student.  After  each  response  the 
computer  analyzed  the  response  and  immediately  provided  feedback.  This 
feedback  was  usually  appropriate  to  the  particular  response  made  by  the 
student.  Both  internal  branching  within  a  frame  and  external  branching  to 
new  material  were  frequently  employed. 
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Four  basic  types  of  training  frames  were  used: 


a.  Lecture  frame:  A  statement  or  series  of  statements  containing 
one  or  more  important  facts  which  did  not  require  the  student  to  make  a 
response . 


b.  Prompted  frame:  A  statement  which  (l)  introduced  only  one  new 
fact,  (2)  required  the  student  to  make  a  relevant  response  to  a  critical 
feature  of  the  new  fact  and  (3)  employed  stimulus  control  to  increase  the 
probability  of  a  correct  response. 

c.  Practice  frame:  A  frame  designed  to  enable  a  student  to  drill 
or  practice  a  terminal  or  criterion  response  without  the  aid  of  a  formal 
prompt.  On  some  of  these  frames  help  options  were  available  which  provided 
on-line  assistance  at  the  request  of  the  student. 

d.  Criterion  frame:  A  frame  designed  to  test  the  effectiveness  of 
the  training  sequence  for  an  objective. 

There  were  very  few  "lecture"  frames  in  which  no  response  was 
required  of  the  student.  Even  the  few  lecture  frames  which  did  occur  were 
usually  constructed  to  produce  a  particular  response  by  the  student. 

Several  types  of  feedback  and  retry  procedures  were  used  in 
conjunction  with  frames  requiring  a  response. 

a.  Fixed  feedback:  The  answer  or  solution,  given  regardless  of 
whether  the  student  responded  correctly  or  incorrectly. 

b.  Confirmation :  A  simple  "Yes”  or  "No",  "Correct"  or  "Wrong" 
type  feedback. 

c.  Explanation :  The  correct  solution  to  finding  the  answer  or  the 
reason  a  response  was  incorrect. 

d.  Simple  retry:  "Try  again." 

e.  Prompted  retry:  Student  was  given  a  hint  or  cue  as  to  the 
correct  answer  and  told  to  try  again. 

Although  errors  are  infrequent  in  a  good  frame,  it  is  important  to 
correct  them  when  they  do  occur.  Thus  the  most  common  type  of  frame  used 
was  a  "prompted  retry"  frame.  If  the  student  missed  the  question  he  was 
given  a  "prompt"  (a  hint  or  explanation)  and  allowed  to  try  again.  When 
possible,  this  feedback  was  designed  to  correct  the  specific  mistake  made 
by  the  student.  In  many  cases  such  feedback  was  added  when  student  tryouts 
disclosed  common  errors. 

3.  Use  of  Training  Media  and  Off-Line  Equipment 

Information  was  presented  to  the  student  in  four  principal  ways. 

A  random  access  image  projector  provided  relatively  fixed  information  such 


as  schematic  circuit  diagrams.  Short  messages  and  questions  were  presented 
on  the  screen  of  a  cathode  ray  tube  (CRT).  In  general,  the  CRT  was  reserved 
for  information  which  could  he  revised  easily  and  quickly  if  needed,  as 
judged  by  student  tryouts.  The  combination  of  the  CRT  and  image  projector 
allowed  precise  control  of  input  information  to  the  student,  as  well  as 
rapid  branching  appropriate  to  the  student* s  progress. 

The  third  source  of  information  to  the  student  was  a  Study  Guide 
booklet.  The  Study  Guide  contained  useful  material  for  the  student  to 
keep  permanently.  This  material  typically  included  lesson  overviews, 
lesson  objectives,  application  problems,  guidance  on  how  to  review,  and 
references.  The  Study  Guide  performed  several  important  functions  for  the 
student:  knowledge  of  the  exact  objectives  which  the  student  was  required 

to  achieve,  a  handy  source  of  job  aids  such  as  math  tables,  elimination  of 
the  necessity  of  taking  notes,  and  brief  but  complete  reference  materials 
needed  for  self  review. 

The  fourth  source  of  information  was  random  access  audio  instructions. 
These  messages  freed  the  student  to  view  and  examine  related  information  on 
the  CRT,  image  projector  and  Study  Guide,  or  to  receive  help  while  operating 
off-line  lab  equipment.  Audio  was  also  used  as  an  alternative  method  of 
presenting  new  material,  reviewing,  or  providing  feedback  for  errors  on  the 
Self  Tests.  During  the  research  on  Long  Term  CAI  (See  Section  III.D), 
instruction  was  made  optional  via  the  CRT  or  via  audio  at  a  number  of  points 
in  the  modules.  It  was  found  that  half  of  the  students  chose  audio  and  the 
other  half  chose  visual  instruction. 

Off-line  training  (operation  of  multimeter  and  oscilloscope, 
identification  of  capacitors,  etc.)  was  controlled  in  two  ways.  Some  of 
the  training  was  done  at  the  CAI  carrel.  The  CAI  media  provided  optional 
help  in  operating  the  equipment.  When  the  student  completed  an  exercise, 
he  entered  his  result  in  the  computer  and  the  computer  provided  tutorial 
feedback  and  instructions.  The  other  type  of  training  was  done  away  from 
the  CAI  carrel.  Here  training  was  controlled  by  requiring  the  student  to 
respond  in  his  Study  Guide  and  check  his  results  by  turning  the  page  or  by 
going  to  the  instructor  in  critical  cases. 

k .  Branching  Techniques  for  Remedial  Training. 

Remedial  methods  were  developed  to  handle  either  specific  instances 
of  deficient  entering  behaviors  or  a  broad  spectrum  of  deficiencies  in  pre¬ 
requisite  skills.  Specific  deficiencies  were  usually  corrected  at  the 
point  where  they  were  important  in  a  lesson.  In  cases  where  it  was  difficult 
to  determine  the  nature  of  the  student's  deficiency,  he  could  be  given 
optional  review  by  means  of  a  review  index  and  allowed  to  correct  his  weakness 
himself.  The  illustration  shown  here  is  an  example  of  optional  remedial 
branching  for  students  with  deficiencies  in  mathematics. 
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Index  for  reviewing: 

Rule 

Practice 

□ 

□ 

Transpose  formulas 

□ 

□ 

Change  prefixes  to  powers  of 

10 

□ 

□ 

Use  powers  of  10  in  formulas 

□ 

□ 

Use  checks  for  correctness 

□ 

I'm  ready  for  an  application 

problem 

The  DC  module  on  series-parallel  circuits  provided  a  case  where 
massive  remediation  was  required.  The  material  was  highly  heirarchical  and 
students  entering  the  module  were  often  unable  to  analyze  simpler  circuits 
or  operate  a  multimeter  correctly  or  test  a  circuit.  Three  branched  remedial 
pretraining  lessons  were  developed  for  this  module.  The  pretraining  was 
designed  to  accomplish  three  things:  allow  students  to  complete  the  pre¬ 
training  in  the  minimum  possible  time  needed,  insure  adequate  performance  on 
the  prerequisites,  and  minimize  the  authoring  time  required  to  develop  the 
materials . 

The  structure  of  the  remedial  lessons  is  shown  in  Figure  U.  The 
student  had  the  option  of  reviewing  any  or  all  of  the  objectives  before 
taking  the  lesson  test.  The  lesson  test  was  arranged  in  a  heirarchical 
sequence  from  basic  to  most  complex  items.  If  the  student  made  an  error 
he  was  sent  back  to  the  corresponding  lesson  instruction  before  continuing 
with  the  lesson  test.  Thus  a  student  could  be  presumed  to  be  competent  on 
the  prerequisites  to  an  objective  before  taking  the  lesson  test  item  on 
that  objective.  The  reason  for  this  procedure  was  that  terminal  performance 
was  quite  complex  in  this  remediation,  and  testing  the  final  objectives  first 
would  make  it  very  difficult  to  isolate  the  reasons  for  student  errors. 

The  effectiveness  of  this  remedial  technique  was  experimentally 
evaluated.  See  Section  III.F.l. 
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Note:  Computation  items  are  normally  completion,  to  prevent  guessing.  If  error 
is  merely  in  decimal  point,  student  may  be  given  prompted  retry. 

Figure  4.  Typical  structure  of  DC  remedial  lessons 
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5*  Control  of  Training  vith  Pretest  Branching 


Two  types  of  pretests  were  developed.  In  Type  I,  pretest  items 
were  given  before  each  objective  to  determine  if  the  student  could  skip 
that  objective.  In  Type  II,  the  pretest  items  with  their  associated  feed¬ 
back  constituted  a  minimal  training  sequence.  If  the  student  failed  to 
meet  certain  criteria  at  any  point,  he  was  branched  to  a  maximum  (detailed) 
training  sequence.  Both  of  these  types  were  empirically  evaluated.  (See 
Section  III.F.2. ) 

a.  Pretest  Design  I  —  Skip  ahead  branching.  This  design  called 
for  the  student  to  be  pretested  on  each  training  objective,  one  at  a  time, 
with  one  or  more  criterion  frames.  Based  on  the  student’s  performance  on 
each  pretest  frame  and  sometimes  on  his  performance  on  an  interrelated 
training  objective  pretest,  branching  decision  rules  determined  whether  he 
was  (l)  branched  ahead  to  the  next  training  objective  pretest  or  (2)  branched 
to  an  instructional  training  sequence  before  being  returned  to  the  next 
training  objective  pretest.  In  other  words,  the  lesson  was  arranged  so  as 
to  determine  in  advance  the  degree  of  familiarity  of  the  student  on  each 
training  objective.  If  a  student  did  not  demonstrate  the  terminal  behavior 
demanded  by  a  training  objective,  the  program  branched  him  to  a  set  of 
instructional  frames  designed  to  teach  that  objective  before  he  could 
continue  with  the  lesson.  If  the  student’s  performance  demonstrated  that  he 
’’knew”  the  training  objective,  he  was  branched  to  the  pretest  for  the  next 
training  objective.  A  flow  chart  model  for  this  pretest  design  is  shown  in 
Figure  5* 


Figure  5*  Flow  chart  model  of  Pretest  Design  I 
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The  writing  of  CAI  lessons  to  fit  this  instructional  design  was 
simpler,  if  not  more  systematic,  than  writing  fixed  sequence  course  material. 
The  basic  procedure  was  to  write  a  separate  sequence  of  instructional  frames 
for  each  training  objective  and  to  conclude  the  sequence  with  one  or  more 
criterion  frames. 

All  students  were  required  to  take  a  short  review  over  all  major 
training  objectives  before  completing  the  lesson  (see  Figure  5)*  This  was 
a  safeguard  against  false-positive  branching  (skipping  one  or  more  training 
sequences  when  in  fact  the  student  did  not  really  know  the  lesson  objectives) 
and  provided  drill  and  practice  repetition  for  over-learning  and  longer  term 
memory. 


b.  Pretest  Design  II  —  Mimimum  vs  maximum  training.  The  pretest 
in  this  design  consisted  of  a  series  of  criterion  questions  testing  all 
major  lesson  objectives.  The  questions  were  arranged  in  a  hierarchial 
learning  sequence  and  programmed  to  provide  confirmation  feedback  as  a 
consequence  of  correct  and  wrong  answers.  At  the  end  of  each  pretest 
question,  branching  decision  logics  determined  whether  the  student  continued 
with  the  pretest  or  branched  to  the  lesson's  maximum  training  sequence, 
fixed  sequence  instructional  frames  covering  all  objectives.  Branch 
decisions  were  based  on  the  student's  cumulative  pretest  performance,  both 
quantitative  and  qualitative.  It  was  possible  for  a  student  to  progress 
through  the  pretest  if  he  did  not  make  too  many  errors  and  if  he  avoided 
distractors  that  were  prejudged  to  be  indicative  of  poor  learning  progress. 

In  other  words,  a  student  was  told  whether  his  answer  to  each  pretest  ques¬ 
tion  was  right  or  wrong.  Wrong  answers  were  weighted  as  to  the  degree  or 
type  of  error.  Decision  branching  logics  depended  upon  error  weight  total. 
The  criterion  for  branching  increased  as  the  student  progressed  through  the 
pretest.  Too  many  errors  or  the  wrong  type  of  error  resulted  in  branching 
the  student  to  the  instructional  frame  sequence  for  all  lesson  objectives. 

The  student  who  missed  only  a  few  pretest  questions  and  avoided  making 
certain  types  of  errors  was  able  to  complete  the  pretest.  Minimal  training 
occurred  if  a  student  avoided  branches  to  the  maximum  training  sequence  and 
completed  the  entire  pretest.  In  this  case  he  either  already  knew  the 
lesson  objectives  before  starting  the  pretest  or  learned  the  objectives 
while  taking  the  pretest. 

A  flow  chart  model  of  Pretest  Design  II  is  presented  in  Figure  6. 
Notice  that  branching  decision  logics  determined  whether  the  student 
continued  with  minimal  training  (pretest  questions  plus  confirmation  feed¬ 
back)  or  branched  to  the  maximum  training  sequence  after  almost  every  pre¬ 
test  question.  All  students  received  an  optional  review  of  selected  major 
objectives  at  the  end  of  the  lesson.  A  unique  feature  incorporated  into  this 
design  was  "Student  Self  Evaluation."  It  was  a  pretest/no  pretest  branch 
decision  based  on  the  student's  response  to  the  question  "Do  you  think  you 
can  solve  for  total  capacitance  in  series  and  in  parallel  circuits?" 
employed  at  the  beginning  of  the  lesson.  Selection  of  the  answer  "Yes"  or 
"Maybe"  resulted  in  branching  to  the  first  pretest  question,  a  "No"  answer 
branched  the  student  immediately  to  the  maximum  training  sequence. 
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Figure  6.  Flov  chart  model  of  Pretest  Design  II 


6.  Student  Control  of  Instructional  Sequencing 

The  student  was  given  the  option  of  skipping  certain  parts  of  the 
instruction  in  a  number  of  training  situations.  These  included  remediation 
of  entering  behaviors,  original  training,  drill  and  practice,  review,  and 
posttest  remediation.  These  option  methods  seemed  to  work  well  provided  the 
student  had  a  clear  knowledge  of  the  objectives  he  was  required  to  achieve. 

Two  important  problems  concerning  optional  branching  were  experi¬ 
mentally  investigated.  The  first  problem  was  whether  extensive  use  of 
optional  branching  would  result  in  a  useful  time  savings  as  compared  to 
relatively  linear  CAI.  An  example  of  extensive  optional  branching  used  in 
Lesson  C  of  the  Series  Circuits  module  is  shown  in  Figure  7*  The  student 
had  separate  options  to  skip  remediation,  training,  practice,  and  review. 
Experimental  results  for  such  training  are  reported  in  Section  III.F.3. 

The  second  problem  was  whether  student  option  works  as  well  as 
program  control  for  bypassing  of  instruction  and  for  posttest  remediation. 

If  so,  it  would  be  preferred  to  program  control  because  instruction  which 
utilizes  student  option  is  much  simpler  to  prepare.  Experimental  findings 
are  given  in  Sections  III.F.5  and  III.F.6. 
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Training  sequence: 
Right  triangles 
Pythagorean  theorem 


Student  self-assessment: 
Sides  of  triangle 


(don 1 1  know ) _ 

. (not  sure) _ 


Training  sequence: 
_ Sides  of  triangle 


10 


Student  self-assessment: 
Find  sin  0,  etc. 


Training  sequence  and 
criterion  problem: 

Use  of  trig  tables  - 
Look  up  sin  0,  et  al 


( don1 1  know ) 


Training  sequence: 

Sin  © ,  cos  0,  tan  0 


13 


Criterion  problem: 

Sin  0,  cos  0,  tan  0 


(error) 


T 


Remedial  Training: 
Sides  of  triangle 
Sin  0,  cos  0,  tan  0 


_ (option) 


(retry 

error) 


(to  new 
problem) 


Dual-control  practice: 
Sin  0,  cos  0,  tan  0 


max  5 


_ (option) 


Optional  practice: 

Look  up  sin  0,  et  al 


T 


max  5 


Training  sequence: 
Use  of  trig  tables 
Inverse  functions 


Review/Practice  Index 


Lesson  Posttest 


_  (options )  _ 


Review  of  training: 

Name  sides 

(10) 

Sin  0,  et  al 

(13) 

Remedial 

(2) 

Practice : 

Find  sin  0,  etc. 

(max  5) 

Look  up  sin  0,  etc 

(max  5) 

Figure  7.  An  example  of  multiple  branching 
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7.  Control  of  Drill,  Practice,  and  Review 


Four  branching  methods  were  successfully  used  for  determining  the 
appropriate  amount  of  practice  or  review  for  each  individual  student.  One 
type  was  strict  program  control.  This  was  used  for  drill  on  skills  in 
which  the  student  had  to  demonstrate  a  predetermined  level  of  competence. 
Computer  programs  were  developed  which  randomly  presented  new  sets  of 
problems.  Drill  was  continued  until  the  student  met  the  criterion.  An 
example  was  drill  on  the  use  of  the  resistor  color  code. 

A  second  type  was  complete  student  option.  This  was  widely  used 
for  review  at  the  end  of  lessons.  It  was  also  used  in  a  number  of  lessons 
to  provide  practice  in  the  middle  of  the  lesson.  Examples  of  these  uses 
are  shown  in  Figure  7  in  the  previous  section. 

Two  kinds  of  dual  control  practice  were  developed.  The  first  kind 
began  with  a  required  criterion  problem.  If  the  student  missed  the  problem 
he  was  required  to  do  another.  If  he  got  the  problem  correct,  he  was 
given  the  option  of  doing  further  practice  problems.  An  example  from 
Lesson  E  of  the  Series  Circuits  module  is  shown  in  Figure  8.  The  second 


Student  directed  to  read 
Study  Guide  explanation 
and  example. 

1 

r 

Student  directed  to  do 
Study  Guide  problem 
and  check  answer. 

'I 

f 

Criterion  problem 
(inductive  circuit) 

(error] ^ 

Practice  problem 
(hints  available) 

1 

^ 

1 

f 

Dual-control 
criterion  problem 
(capacitive  circuit) 

(error)  ^ 

Dual-control 
practice  problem 
(hints  available) 

^  (correct) 

(error) 

(option)  ^ 

Practice  problem 
(hints  available) 

^  - 

(To  next  objective  or  Self  Test) 

Figure  8.  An  example  of  dual  control  practice. 
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type  of  dual  control  practice  occurred  in  a  review  lesson  covering  a  number 
of  objectives.  Each  objective  began  with  optional  practice  problems  followed 
by  a  required  criterion  frame.  If  the  student  missed  the  criterion  frame 
and  had  skipped  the  practice,  he  was  required  to  go  back  and  do  the  practice. 
Both  of  these  methods  of  dual  control  were  experimentally  compared  to  fixed 
sequence  practice.  See  Sections  III.F.3.  and  III.F.l*. 

C.  Data  Management  System 

During  the  course  of  the  project,  a  complete  data  management  system  was 
developed  and  evaluated.  The  system  performed  three  main  functions:  (l) 
identification  and  classification  of  frames  and  student  responses,  (2) 
monitoring  of  student  progress  and  achievement,  and  (3)  data  analyses  and 
summaries  useful  for  evaluation  and  revision  of  CAI  courses.  Components  of 
the  system  were  revised  several  times  on  the  basis  of  experience  with  course 
operation  and  revision.  Details  of  the  final  versions  are  given  below. 

1 .  Frame  and  Response  Identifiers 

The  frame  identifier  was  a  10  digit  code  which  provided  a  unique 
frame  label  for  branching  and  documentation,  identified  the  lesson  and 
exact  objective,  and  classified  the  type  of  instruction  used  in  the  frame. 

A  description  of  the  frame  identification  code  is  given  in  Table  1  and  an 
example  of  use  of  the  code  in  a  frame  is  given  in  Figure  9* 

The  first  four  digits  provided  a  label  for  the  frame.  The  first 
digit  was  a  letter  indicating  mainline  ( V),  branch  sequence  ("a,"  "b," 

"c,"  etc.),  pretest  ("p")  or  lesson  test  ("t")  frame.  The  next  three  digits 
indicated  numerical  sequence  of  the  frame  within  the  lesson  and  was  unique 
to  each  frame.  The  last  six  digits  were  used  to  identify  the  frame  as  to 
lesson,  training  objective,  frame  type,  principal  consequence  of  a  wrong 
answer,  special  usage  and  branching,  respectively. 

The  frame  in  Figure  9  is  from  branch  sequence  "an  of  lesson  "1," 
objective  "e."  It  is  frame  type  "V!  (optional  hint  available).  The 
principal  consequence  of  a  wrong  response  is  a  prompted  retry  "p”:  Notice 
that  under  response  type  !tp9"  (the  first  unanticipated  response)  the  student 
is  given  a  prompt  before  retrying. 
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TABLE  1 


Frame  Identifiers  and  Descriptions 


Column 


Identifier  Description 


1 

2-4 

5-6 

7-8 

9-10 


Mainline,  Branch  Sequence,  Pretest  or  Test  Frame 
Three  digit  unique  number  (sequence  based) 

Lesson  and  Training  Objective 
Frame  Type  and  Consequence  of  WA 
Special  Author  Usage 


Column  2-4:  Unique  Frame  Number.  Assigned  to  each  frame  and  based 

on  sequence  or  position  in  lesson,  i.e.,  000,  001,  002, 
. ..,  356,  ...,  999. 


Column  5 i  Lesson  Identifier.  A  single  letter  or  number  to  uniquely 
identify  each  lesson  within  a  module.  Usually  based  on 
sequence,  i.e.,  A,  B,  C,  ...  or  1,  2,  3,  ... 

Column  6:  Training  Objective.  A  single  letter  or  number  used  to 

uniquely  identify  each  lesson  training  objective,  i.e., 
A,  B,  C,  ...  or  1 ,  2 ,  3,  ... 


Column  7:  Frame  Type.  Code  and  types  shown  below: 


0  Information/Directions 

1  Pretest 

2  Lecture 

3  Prompted 

4  Optional  hint 


8  Practice 

5  Criterion 

6  Review  criterion 

7  Lesson  test 

9  Supplemental  test 


Column  8:  Consequence  of  Wrong  Answer.  One  digit  code  to  identify 
the  principal  behavioral  consequence  (retrys,  feedback, 
or  branch)  for  a  student  making  a  wrong  answer  to  a 
frame.  Code  and  consequence  listed  below: 


r  simple  retry 
p  prompted  retry 
e  explanation 
a  answer 
c  confirmation 


t  thematic 

d  delayed  (frame  type  7  or  9 ) 
u  unspecified  (frame  type  0  or  2) 
b  branch  (frame  type  l) 


Column  9  and  10:  Special  Author  Usage.  These  columns  were  available 

for  special  author  usage.  Commonly  used  for  a  code 
"B"  to  indicate  frame  programmed  for  conditional  or 
unconditional  branch. 
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AUTHOR-CODER  FRAME  PLASHING  GUIDE 


PRR  PR  X 


FRAME  IDENTIFIER  SPECIAL  IDENTIFIERS/ ADDRESSES 


a  360 

(l-U  :  Label) 

(Author ) 

( Coder ) 

le 

(5-6  :  Lesscn-objective )  Film 

4p 

(7-8  :  Frame  type-consequence)  Audio 

PO 

(  9  :  Learning  task;  10: 

Graphic 

SHUTTER:  Open _  Close  X 

(CRT  Text  +  feedback:  11  3-row  lines;  max  l6  2-row  lines) 


0  5  10  20  30  39 

lit  *  *  1 

37.  Type  h  if  you  need  a  hint. 


(P-K-C) 


RESPONSE  TYPE/RESPONSE/ (Special  Instructions) 


BRANCH 


FEEDBACK 

0  5  10  20  30  39 

III  I  I  I 

hi /h 

Reduce  it  to  a  simple  parallel  circuit 
by  finding  the  total  resistance  in 
the  left  leg. 

21/3.33 

On  the  button.  C 

p9/unl 

The  4  and  6  ohm  resistors  are  in  series 
so  their  resistance  is  4  +  6  =  10.  Now 
you  have  a  simple  parallel  circuit. 


TO 


re 

[retry] 

nf 

[next  frame] 


re 


e9/un2 

Rt  =  1/10+1/5  \j 


3.33 


C 


nf 


Figure  9.  Example  of  frame  and  response  identification 
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Actual  student  responses  and  their  consequences  were  identified  by 
a  two  digit  code.  A  list  of  codes  for  each  digit  are  shown  in  Table  2. 

The  first  digit  indicated  whether  the  response  was  judged  as  correct  (a 
number  from  1  to  7 )  or  wrong  (a  letter).  The  particular  number  or  letter 
indicated  the  type  of  feedback  programmed  to  be  a  consequence  to  that 
particular  response.  A  special  category  of  zero  ("0")  was  used  to  indicate 
special  frame  usage.  The  second  digit  of  the  response  identifier  could  be 
used  to  indicate:  (l)  type  of  special  frame  response  to  "0"  first  digit 
code,  (2)  position  of  a  correct  or  wrong  penlight  answer,  (3)  keyboard 
answer  choice  "l"  to  "6"  or  (b)  unrecognized  keyboard  answer  "9"-  Examples 
of  use  of  the  response  identifiers  are  given  in  Figure  9- 


TABLE  2 

Response  Type  Identifiers 


1 .  Sre?ial  Response  Tyres 


Column  I 

0 

0 

0 

0 


Column  2 
0 

1-6 

7 

8 


Identification  Description 

Unspecified  response,  latency  recorded 
Student  option  choice  (position) 

Backup  option 
Light  pen  mismatch 


2.  Correct  or  Wrong  Responses  -  Column  1  Identifier  Codes 


Column  I 

CA  WA 

1  or  a 

2  or  c 

3  or  e 

4  or  t 

5  or  u 

6  or  b 

7  or  d 

r 

P 

3.  Correct  or  Wrong  Responses 


Consequence  Identification 


Answer 

Confirmation 

Explanation 

Thematic 

Unspecified 

Branch 

Delayed  Feedback 
Retry 

Prompted  retry 
-  Column  2  Identifier  Codes 


Column  2 

"lM  to  ”6”  Lightpen  or  objective  constructed  response  position 

,,9"  Unrecognized  (wrong)  constructed  response 
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2.  Monitoring  of  Student  Progress  and  Achievement 


Special  proctor  messages  were  developed  which  enabled  the  instruc¬ 
tional  proctor  to  continuously  monitor  the  student  while  he  was  on-line. 
These  messages  could  be  used  to  determine  if  the  student  was  progressing 
too  slowly,  to  determine  if  his  test  performance  was  unsatisfactory,  or  to 
provide  personal  tutoring  if  necessary  on  the  particular  objectives  with 
which  the  student  was  having  difficulties. 

Examples  of  Proctor  messages  are  given  in  Figure  10.  For  each 
student  the  messages  contained  beginning  times  for  lessons,  remediation, 
and  lesson  (self)  tests;  time  on  lessons,  remediation  and  self  test;  self 
test  scores  and  particular  wrong  items  on  self  test. 


Station 

06  mp06dc 

proctor  message 

Begin  lesson  DC -13,  1^35  hr. 

Station 

lh  maOUact 

proctor  message 

ACT  6 

wrong  answers 

3  1*  11 

score  8l 

ACT-6  lesson  time  56 

begin  self  test 

8U0hr  self  test 

time  6  self  test 

Station 

08  sn66scr 

proctor  message 

SCR-UlA 

wrong  answers 

2  u 

End  12U5  On  8 

Score  Q0% 

SCR-Ul 

Beg  Rem  1226 

End  1236  On  10 

Beg  altst  1237 

Station 

12  mal2act 

message  code  10 

[sign  off] 

Figure  10.  Examples  of  proctor  messages  used  for  monitoring  of  student 
progress  and  achievement 


3.  Student  Data  Analysis  and  Summaries 

The  purpose  of  the  data  programs  was  to  make  it  possible  for  course 
authors  to  quickly  and  accurately  pinpoint  specific  weaknesses  in  lesson 
material  and  identify  the  reasons  for  these  weaknesses.  Five  types  of 
programs  were  developed  which  allowed  the  author  to  probe  each  lesson  to 
the  level  of  detail  needed  to  revise  it: 

a.  Lesson  summary.  This  was  the  most  general  type  of  analysis. 

It  showed  average  performance  by  each  student  on  each  type  of  frame  as  well 
as  the  average  for  all  students.  An  example  is  shown  in  Figure  11.  Average 
performance  on  the  lesson  (self)  test  was  somewhat  low  (7Q%  in  col.  7)-  One 
reason  appears  to  be  that  the  low  scoring  students  (MP26  and  MP28)  were  not 
helped  by  the  optional  review  (col.  6). 

b.  Summary  by  objective.  This  was  similar  to  the  Lesson  Summary 
except  that  the  analysis  was  made  for  each  separate  objective.  See  Figure 

12.  From  this  summary  the  author  could  immediately  determine  the  effectiveness 
of  instruction  on  each  objective. 
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rnnp_  LESSON  SUMMARY  (COURSE/STUDENT/FRAMETYPE) 

T  =  TOTAL  RESPONSES  ON  FIRST  TRY<00-99) 

...  -  _  P*PEPCFNT.CORF£CT..ON.FIRST_TRYJOOO-100I _ _ _ 

L*RFSPONSE  AVFRAGE  LATENCY  in  SECONDS  1000-999 1  PAGE  02 

_ X-F 1ELD_QYLR£LQU _ _ 

*0*  1*  2#  3*  4*  5*  6*  7*  8 

—  COR-SEG  *USCR*  CHOICE  *  PRETEST  _ * _ LECTURE,  _* _ PROMPT  —  *  OPT-HINT#  -CRITERION*  .REVIEW  »  SELF  TEST  *  PRAC I  I CE 

DC  013  *  NO  *  T  P  L  *  T  P  L*TP  L*TP  L«TP  L*TP  L*TP  L*TP  L*TP  L 

*mp23«06  100  000*11  063  042*  *  *10  020  026*02  100  018*03  066  008*10  090  026*01  100  0*1 

“•  M p 2 4 » 0* 7~ 0 7 r  04 3 *  11  0 6 3  0 32*  ♦  *01  100  123*01  100  008*  *10  100  041*01  100  007 

*MP?5*06  100  037*12  058  021*  *  *12  050  019*01  100  008*  *10  090  026*01  100  011 

“*VP26*06  100  067*13  030  072*  "  '“  *_  *“  #03  100  035*03  066  058*10  060  077*01  OOO  030 

*  M  P 28*06  1 0_0  0 5 6 *  11  054  ~04 2*  T  *T7~020  069*02  100  035*03  066  022*10  070  0  37*0l” 100  C14 


* v E A N * 0 6  091  014*11^057  004*  *  *08  027  003*01  087  028*03  060  029*10  078  00**01  068  020 


Figure  11.  Summary  of  individual  and  mean  performance 
on  each  type  of  frame 


CoDE_  SUMMARY  BY  OBJECTIVE  ( ST U DENT/ OBJECT  I VE/FRAME  TYPE) 

T-ToTAL  RES°CNSES  ON  FIRST  TR Y ( 00-99 ) 

P*PERCENT  CORRECT  ON. FIRST  TRY(OOO-IOO) 

L-RESPONSE  AVERAGE  LATENCY  IN  SECONDS ( 000-999 )  PAGE  11 

X-FIELD  OVERFLOW 
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*05 

100 

010*02 

100 
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100 
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026*02 

100 

022* 

*05 
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016*02 

100 

030*01 

100 

021* 

•\PU6»U1  100 
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050 

024*0r 

100  064*01 

TOO 
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*04 

100 

"012*02 

050 

030*01 

100 

042* 

—  - 

*  — 

*XP1 3*0 1 

100 

011*01 

000 

025*03 

100 

039*02 

100 
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100 
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100 

026* 
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000 
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013*02 
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019*01 

100 
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»XP16*01 

100 

034*01 

000 

036*03 

100 

032*02 

100 

024* 

*05 

100 

014*02 

100 

010*01 

100 

016* 

-■ 

*XP17*01~100 

010*01 

000 

010*03 

100 

061*02 

100 

022* 

*05 

100 

011*02 

100 

013*01 

100 

015* 

*XP1 8*01 

100 

028*02 

100  031*01 

100 

074*01“ 

100' 

040* 

*04 

100 

032*02'  050 

022*01 

loo 

022* 

SCP  005 

OBJ-  I 

*ME AN*0 1 

100 

024*01 

021 

036*02 

100 

033*01 
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024* 
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091 

0 14*02 
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030* 

Figure  12.  Summary  of  individual  and  mean  performance  on  each 
objective,  sorted  by  frame  type 
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c.  Detailed  frame  analysis.  If  an  objective  was  not  being  taught 
properly,  the  author  could  examine  performance  of  individual  students  on 
each  frame.  Two  types  of  analysis  were  used.  The  example  in  Figure  13 
shows  frames  sorted  by  objective  so  that  the  frames  pertinent  to  each 
objective  could  be  found  quickly  and  compared.  Notice  that  five  students 
made  errors  (letter  in  col.  1  of  response  type)  on  the  pretest  frame ,  P315 5 
and  were  given  further  training.  Three  of  these  five  students  made  errors 
on  their  first  try  on  criterion  frame  M320  but  all  were  correct  on  criterion 
frame  M330. 


SCP  0009  (OBJECT  |  VE-FRAME  TYPE  SEQUENCE  >  11  10  71 _ PAGE _ A  7 


frame-  stuoEnT  response  aec  latency  response  ~  '  ti^e 

TYPE  SEO  (SEC)  24 hr 
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1-FRAME  TYPE _ 
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29 
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05 
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13 
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6 
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C  3 

66 
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11 
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20 
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24 

58 

18.0 
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10 
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29 
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XP1  4 

C2 
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1  1 
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10 
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XP1  7 

“  24 

19 

34.  e 
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11 
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29 
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24 

45 
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29 

1  13J 

*P16 

?  4 

1 9 
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29 
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XP15 

2  4 

18 

14.5 

POW 

10 
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29 
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xP  l 8  ~ 

C  3 

*  *  1? 

66.9 

ROW 

10 
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20 

1002 

3-FRAME  TYPE 

1? 

STUOFNT  s 

—  AVERAGE 

LATENCY 

-FIRST  TRY 

38.8 

M320JH5P 


NP02 

PI 

40 
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12 
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8 

1453 

NP02 

22 

41 
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ROW 

12 

COL 

14 

1453 

NP06 

22 

12 
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POw 

12 

COL 

14 

1335 

NP03 

P  3 

67 

6.2 

ROW 

1? 

COL 

20 

1337 

NP0) 

22 

68 

1 5 .0 

ROW 

12 

COL 

14 

1337 

X  P  1  4 

PI 

47 

9.8 

ROW 

12 

COL 

8 

1009 

xpi* 

22 

48 

85.0 

ROW 

13 

COL 

14 

1309 

XP18 

22 

20 

46.2 

ROW 

12 

COL 

14 

1003 

3  Gtuoents 

—AVERAGE 

LATENCY 

-FIRST  TRY 

18.6 

M3  30 JH  5P 

.....  - 

NP02  22 

42  -  5.4 

ROW 

12 

COL 

14 

- - - - - 

1454 

NP06  22 

13  5.4 

ROW 

12 

COL 

14 

1335 

"NPOr  22 

69  1.8 

ROW 

12 

COL 

14 

1)37 

XP14  22 

49  1.0 

ROW 

12 

COL 

14 

1009 

>PH  22  - 

21  20.3 

ROW 

12 

“COL 

14 

1003 

3  STU0ENTS— 

AVERAGE  LATENCY 

-FIRST  TRY 

6.7 

Figure  13.  Detailed  performance  of  each  student  on  each  frame, 
sorted  by  objective  and  frame  type 

The  second  type  of  detailed  frame  analysis,  preferred  by  some 
authors,  simply  listed  the  frames  in  numerical  sequence,  rather  than  by 
objective  and  frame  type. 

d.  Student  trail.  This  listed  all  the  responses  of  the  student  in 
the  order  in  which  they  occurred  (see  "rec  seq"  in  Figure  13)  so  that  the 
path  of  the  student  through  the  lesson  could  be  easily  followed.  Information 
included  for  each  response  was  the  same  as  for  the  frame  analyses  (frame, 
response  type,  latency,  etc.). 
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D.  Course  Revision  Methodology 


1.  Student  Tryouts 

Student  tryout  runs  were  made  on  each  CAI  module  before  it  was 
operationally  tested.  During  the  student  tryout  phase,  selection  of  students 
was  made  to  insure  coverage  of  the  aptitude  range,  but  no  formal  sampling 
procedures  were  used.  Generally,  more  information  was  obtained  from  the 
intermediate  or  lower  achieving  students  than  from  the  high  achieving  students. 

Tryouts  provided  data  to  evaluate  training  effectiveness  and  to  act 
as  a  guide  for  revision:  (l)  performance  on  CAI  lessons  and  lesson  test; 

(2)  performance  on  CAI  supplemental  test,  and  (3)  performance  on  School 
Area  Examination. 

One  of  the  major  advantages  of  a  CAI  system  is  its  ability  to  process 
large  amounts  of  student  data.  Thus  it  is  possible  to  make  very  effective 
revisions  of  a  course  because  defects  in  the  instruction  can  be  exactly 
pinpointed.  The  computer  keeps  a  continuous  record  of  all  student  responses 
during  training  as  well  as  performance  on  the  posttests.  If  students  per¬ 
form  poorly  on  particular  posttest  questions,  the  trouble  can  be  isolated 
to  such  things  as  inadequate  early  training,  insufficient  practice,  or 
poor  retention  because  of  insufficient  spacing  of  practice. 

2.  Use  of  Tryout  Performance  Data 

Responses  made  by  each  student  on  each  frame  of  a  lesson  were 
recorded,  and  a  printout  of  these  data  provided  the  major  guide  for  all 
revisions . 

The  revision  analysis  began  with  the  terminal  performance  on  each 
training  objective  tested  on  the  School  Area  Examination  and  the  Supplemental 
Test,  because  these  tests  measured  long  term  retention  and  were  used  for 
evaluating  training  methods  (see  Figure  l4). 

If  the  error  rate  on  a  particular  training  objective  was  found  to 
be  greater  than  10-15$ >  a  step-by-step  backward  analysis  was  performed  to 
isolate  the  lesson  and  specific  frames  responsible  for  the  training:  (l) 
performance  on  lesson  test,  (2)  performance  on  criterion  frames  within  the 
lesson,  and  (3)  performance  on  individual  training  frames  of  the  objective. 
Analyzing  backwards  enabled  authors  to  determine  whether  or  not  a  trainee 
had  ever  demonstrated  satisfactory  performance  on  a  training  objective  in 
the  lesson  test  or  criterion  frames.  If  not,  extensive  revision  was  necessary 
to  improve  the  instructional  content  of  the  lesson.  On  the  other  hand,  if 
the  trainee  passed  the  lesson  test  items  or  criterion  frames,  it  showed  the 
need  to  include  additional  drill  and  practice  in  order  to  promote  long-term 
memory  and  success  on  the  Area  Examination.  Another  critical  analysis  was 
to  examine  failed  test  items  or  frames  for  proper  wording,  adequacy  of 
distr actors,  and  format  of  presentation. 
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EXIT 


REVISION 

A  SUCCESS  HERE  AND  FAILURE  ON  AREA 
TEST  SHOWS  NEED  FOR  DRILL  AND 
PRACTICE  TO  STRENGTHEN  LONG  TERM 
MEMORY - REVISE  ACCORDINGLY. 

B  REWORD,  CHANGE.  AND/OR  RESTRUCTURE 
FORMAT. 

Ci  FAILURE  ON  AREA  TEST  AND  OR 

1  SELF-TEST  INDICATES  STUDENT  KNEW 
TERMINAL  BEHAVIOR  AT  THIS  POINT 
8UT  DID  NOT  RECEIVE  SUFFICIENT 
PRACTICE - REVISE  ACCORDINGLY. 

C9  SHOWS  THAT  TRAINING  FRAMES  AND/OR 
^  THE  INSTRUCTIONAL  STRATEGY  ARE 
DEFICIENT.  REQUIRES  MAJOR 
REVISION  OF  ENTIRE  TRAINING  SEQUENCE 
AND  INSTRUCTIONAL  FRAMES. 


Figure  l4.  Flow  chart  of  sources  of  data  and  types  of  revisions  indicated 
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Team  leaders  often  assisted  authors  by  recommending  or  suggesting 
revisions  based  on  performance  data  analysis.  Examples  of  analysis  of 
tryout  data  and  suggested  revisions  are  shown  in  Figure  15. 


Example  1.  REVISIONS  OF  MAJOR  TOPICS 


1.  Add  new  objectives; 
delete  1  objective 


2.  Teach  average  value 
before  RMS  value  to 
take  advantage  of 
students  previous 
experience;  add 
discrimination 
frames  in  which 
student  compares 
various  concepts 


Example  2.  ANALYSIS  OF  A  POORLY  TAUGHT  OBJECTIVE 

Objective:  Select  from  a  list  the  definition  of  angular  velocity 


Data 

%  Error 

Errors 

Analysis 

Area  Test  Q. 

A15 

28 

a  ,d 

Confusion  with  flux  lines  cut  per  unit  time; 

Cl  5 

60 

a  ,a  ,d 

question  refers  to  angular  velocity  of 

D15 

57 

b,b  ,b,d 

s in  ewave . 

Crit  Test  Q. 

U. 

50 

c,c,c,c. 

Confusion  with  number  of  radians  in 

c  ,c,c  ,c  , 

one  cycle;  question  refers  to  angular 

a  ,a  ,a 

velocity  of  rotating  object. 

Training  Frames 

m060X 

— 

m060x  and  m060Xl  are  reading  frames;  m060 

m060Xl 

— 

is  only  response  frame  and  deals  with  rotation 

m060 

10 

b,b 

mOTO 

— 

m070  deals  only  indirectly  with  def.  of 

angular  velocity  of  sinewave. 

Suggested  revisions:  Need  response  frames  on  definition  of  angular  velocity 
for  rotation  and  also  for  sinewave.  Need  discrimination  frame  re  flux  lines 
cut  per  unit  time,  radians  per  cycle,  and  angular  velocity. 


Difficulty 

1.  Low  scores  on 
particular  area 
test  questions 


2.  Many  students  take 
lesson  7  over;  low 
scores  on  pertinent 
criterion  and  area 
test  questions 


1 .  Not  taught 
(discrepancy  between 
CAI  and  school 
objectives ) 

2.  Interference  between 
highly  similar  concepts 


Figure  15*  Examples  of  data  analysis  and  suggested  revisions 
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An  effective  revision  process  makes  it  possible  to  solve  two 
important  training  problems.  One  problem  is  determining  the  amount  of 
training  and  practice  required.  For  instance,  excessive  practice  merely 
wastes  the  time  of  the  brighter  students.  Furthermore,  if  a  program  is 
too  long  to  begin  with,  it  is  often  difficult  to  find  out  if  it  could  be 
shortened  effectively.  A  second  problem  is  determining  the  remedial  branch¬ 
ing  sequences  which  should  be  included.  It  is  clearly  not  worthwhile  to 
prepare  branching  materials  which  students  never  use.  For  example,  the 
method  of  solving  the  above  problems  for  the  AC  CAI  Theory  Module  was  to 
prepare  an  initial  version  of  the  course  consisting  of  a  minimal  instruc¬ 
tional  sequence.  The  module  was  then  given  several  student  tryouts.  After 
each  tryout,  the  lessons  were  revised  and  additional  training  added  as 
needed.  Most  of  the  remedial  branching  and  optional  student  review  used  in 
the  AC  module  were  added  as  a  result  of  student  tryouts. 


III.  Experimental  Evaluation  of  CAI  Training 
A.  General  Procedures 


1.  Selection  of  Students 


A  form  of  systematic  randomization  was  used  to  select  students  for 
the  tryout  and  revision  steps  during  the  development  of  the  CAI  modules. 
Because  only  a  few  students,  10  to  20,  were  used  for  each  tryout,  a  restric¬ 
tion  in  selection  was  imposed  to  insure  that  aptitude  levels  were  proportional 
to  that  of  the  school  population. 

All  students  who  participated  in  the  final  operational  evaluation  of 
CAI  materials  were  randomly  selected  from  class  rolls  provided  by  the  Basic 
Electricity/Electronics  School.  The  only  restriction  was  that  students  had 
to  have  completed  all  classroom  instruction  immediately  proceeding  the  CAI 
materials. 

2.  Number  of  Students  Used  in  Project 

A  total  of  760  students  were  used  during  the  tryout  and  revision 
stages  of  CAI  module  development,  operational  tests  of  individual  modules, 
evaluation  of  effects  of  long  term  CAI,  and  special  studies  of  branching 
techniques  and  paired  student  training  (see  Table  3). 
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TABLE  3 


Number  of  Students  Used  in  Project 


Use  Category 

Number  of  Students 

Module  Tryouts  2k6 
Operational  Tests  175 
Special  Studies 

Long  Term  CAI  50 
Student  Opt ion /Program  Controlled  Training  96 
Student  Controlled  Remediation  108 
Paired  Student  Training  75 

Total  760 


3-  Student  Terminal  Hours  Used 


A  total  of  760  different  students  received  CAI  training  during  the 
project  and  used  more  than  10,197  student  hours  of  terminal  time.  Table  k 
presents  a  breakdown  of  these  data. 


TABLE  U 

Student  Terminal  Hours 


Category 

Students 

Total  Student  Terminal  Hours 

Student  Tryouts 

2U6 

3,277 

Operational  Tests 

175 

1,89U 

Special  Studies 

339 

5,026 

Grand  Totals 

760 

10,197 

h .  Data  Collection  and  Analysis 

Three  types  of  data  were  collected,  recorded  and  used  in  the 
experimental  evaluation  of  CAI  training. 

a.  Background  measures.  School  records  were  the  source  of  aptitude 
test  scores — Electronics  Technician  Selection  Test,  General  Classification 
Test,  and  Arithmetic  Test — on  each  student  for  use  in  correlational, 
predictive  analyses  and  for  potential  covariants  in  analysis  of  variance 


between  CAI  and  classroom  controls  performance  measures.  School  records 
also  provided  data  on  previous  school  performance.  During  the  project, 
previous  school  performance  proved  to  be  the  best  predictor  of  CAI  perfor¬ 
mance  (,r  =  .53  to  .76). 

b.  CAI  time  measures.  Three  major  time  measures  were  recorded  for 
each  student  on  each  CAI  module.  Training  time,  computer  recorded,  was  the 
total  time  required  for  a  student  to  complete  each  lesson  and  lesson  tests. 
Module  time  consisted  of  total  time  on  the  system  from  sign-on  until  sign- 
off,  i.e.,  cumulative  attendance  time  for  each  student  each  day  on  the 
module.  Time  in  other  activities,  obtained  by  subtracting  training  time 
from  module  time,  was  the  time  spent  by  a  student  reviewing  for  lesson  tests 
and  on  rest  breaks  between  lessons. 

c.  CAI  test  performance  measures.  Three  kinds  of  test  scores  were 
recorded  during  CAI  training:  (l)  Lesson  Test  Scores,  (2)  Supplemental  Test 
Score,  and  (3)  School  Area  Test  Score. 

d.  Statistical  analyses.  The  standard  procedure  for  summarizing 
student  data  descriptively  and  for  inferential  comparisons  with  control 
groups  was  to  record  all  data  for  experimental  and  control  students  on 
computer  punch  cards.  These  cards  were  then  run  on  various  programs  to 
obtain  descriptive  statistics,  multiple  correlations  and  inferential 
comparisons  (analysis  of  variance  and  analysis  of  covariance). 

B.  Effectiveness  of  Course  Revision  Methodology 

The  process  of  student  tryouts  and  revisions  proved  to  be  an  extremely 
important  step  in  developing  effective  instruction.  First,  and  most  critical, 
tryouts  provided  data  to  evaluate  general,  as  well  as  specific,  training 
effectiveness  of  the  CAI  course  materials.  Data  analysis  pinpointed  weak¬ 
nesses  in  the  instruction  of  each  training  objective  and  indicated  both  the 
location  and  type  of  revisions  needed.  Second,  it  was  not  possible  to 
anticipate  student  deficiencies  in  prerequisite  skills  and  knowledges 
assumed  in  the  original  development  of  CAI  materials.  Student  tryouts  were 
used  to  determine  points  in  the  instruction  where  remediation  was  needed. 

Third,  the  tryout  and  revision  cycle  provided  a  useful  way  to  aid  inexperienced 
authors  in  learning  how  to  prepare  good  instructional  materials.  Finally, 
even  experienced  instructors  made  oversights  and  other  minor  errors  when 
preparing  the  highly  detailed  material  required  for  individually  adaptive 
instruction.  These  mistakes  were  easily  discovered  by  testing  the  lesson 
with  a  few  students.  This  part  of  the  report  is  devoted  to  presenting  the 
results  of  revisions  on  the  CAI  modules  of  AC  Theory,  Inductance,  Capacitance 
and  Series  Circuits  to  illustrate  the  effects  of  revisions  on  lesson  times 
and  scores  as  well  as  on  total  module  training  times  and  examination  scores. 

1.  Revisions  of  AC  Theory  Lessons 

The  effectiveness  of  the  revision  process  is  illustrated  in  Table  5« 
These  were  lessons  which  were  excessively  long  in  comparison  to  the  other 
five  lessons  in  the  module.  Training  time  was  reduced  by  more  than  one- 
third  from  the  second  to  the  third  tryout  of  AC  Theory. 


35 


TABLE  5 


Median  Performance  on  Three  "Slow"  Lessons 


Tryout  N  Lesson  Time  (Min.)  Test  Score  (%) 

Second  15  59  91 

Third  22  37  95 


2.  Revisions  of  Inductance  Lessons 


Revision  effects  in  lesson  time  and  lesson  test  scores,  between  the 
first  student  tryout  and  the  Final  Evaluation,  are  shown  in  the  last  column 
of  Table  6.  These  changes  reflect  an  average  of  three  revisions  per  lesson. 
Note  that  revisions  did  not  always  result  in  both  a  savings  in  lesson  time 
and  an  increase  in  test  score  (Lessons  1  and  7)-  It  is  also  important  to 
point  out  that  the  effects  of  certain  kinds  of  revisions  are  not  always 
reflected  in  either  lesson  time  or  score  measurements  (Lesson  5)-  In  the 
case  of  Lesson  5 ?  the  last  revision  included  planning  a  new  instructional 
strategy,  rewriting  almost  all  the  lessons  instructional  content  and 
drastically  changing  the  lesson  test  because  of  an  original  failure  to  place 
the  proper  emphasis  in  the  lesson  and  lesson  test  upon  the  more  important, 
if  not  difficult,  training.  This  meant  that  previous  measures  became 
inappropriate  for  evaluating  the  true  effect  of  the  final  revision  effort 
on  this  lesson. 


TABLE  6 

Revision  of  Inductance  Module  Lessons 


Lesson 

Measure 

First 

Tryout 

Final 

Eval. 

Changes  Between  First 
and  Final  Revision 

mean  time  (min.) 

100 

62 

38%  savings 

mean  score  {%) 

85 

88 

k%  increase 

mean  time 

101 

TT 

23%  savings 

mean  score 

81 

90 

11%  increase 

mean  time 

89 

5U 

38%  savings 

mean  score 

80 

86 

8%  increase 

mean  time 

65 

77 

Not  applicable 

mean  score 

80 

79 

Not  applicable 

mean  time 

16 

16 

No  change 

mean  score 

80 

96 

20%  increase 
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3.  Revisions  of  Capacitance  Lessons 


Significant  changes  in  several  sample  lessons  between  the  first 
student  tryout  and  the  Capacitance  Module  operational  test  are  shown  in 
Table  7*  Dramatic  changes  on  Lesson  7  measures  were  due  to  major  revisions 
of  the  entire  instructional  content;  the  lesson  test  was  not  revised. 


TABLE  7 

Revision  of  Capacitance  Module  Lessons 


ssson 

Measure 

First 

Tryout 

Cap. 

Op-test 

Revision  Results 

i 

mean 

time 

(min. ) 

27 

18 

40%  savings 

_L 

mean 

score 

(?) 

97 

98 

No  change 

2 

mean 

time 

46 

36 

21%  savings 

mean 

score 

97 

93 

4%  increase 

c 

mean 

time 

73 

56 

22%  savings 

mean 

score 

86 

84 

2%  increase 

*7 

mean 

time 

91 

48 

47%  savings 

1 

mean 

score 

74 

86 

16%  increase 

Module 

Test 

(mean 

score ) 

82.2 

88.6 

8%  increase 

A  unique  revision  step  in  the  Capacitance  Module  was  the  addition 
of  another  lesson,  Lesson  10.  It  was  introduced  to  serve  as  a  review 
lesson  and  included  drill  and  practice  problems  on  major  training  objectives 
from  each  of  the  original  nine  lessons.  The  lesson  did  not  contain  a  test, 
since  no  new  objectives  were  introduced.  The  mean  lesson  time  for  Lesson  10 
was  110  minutes,  but  this  time  increase  was  compensated  for  by  time  savings 
gained  from  revision  of  the  other  lessons.  Total  effect  of  revisions  of  the 
Capacitance  Module  was  perhaps  best  shown  by  increases  in  the  mean  score  on 
the  Module  Test  between  the  first  tryout  and  the  operational  test,  82.2% 
to  88.6%,  respectively. 

It  is  important  to  report  that  only  two  revisions  of  the  Capacitance 
Module  lessons  were  necessary  between  the  first  student  tryout  and  the 
operational  test  for  the  module.  This  increase  in  efficiency  was  thought  to 
be  due  to  the  fact  that  all  lesson  authors  had  received  previous  experience 
in  writing  and  revising  lessons  during  development  of  the  Inductance  Module. 
From  the  first  student  tryout,  capacitance  lesson  test  scores  were  very  high 
and  the  main  aims  of  revisions  were  to  smooth  instructional  flow  and  to 
attempt  to  increase  lesson  time  savings, 

4.  Revisions  of  Series  Circuits  and  Resonance  Module 


Four  tryouts  and  revisions,  using  a  total  of  44  students,  were  made 
on  the  Series  Circuits  and  Resonance  Module,  The  effects  of  these  revisions 
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are  shown  in  Table  8.  Total  module  training  time  was  reduced  by  five  hours 
(19%)  despite  the  fact  that  an  extra  lesson  was  added  to  the  module  and 
the  Supplemental  Test  Score  mean  was  increased  by  6%. 


TABLE  8 

Revision  of  Series  Circuits  and  Resonance  Module 


Tryout 

N 

Module  Time 

Supplemental 

Test  Score  (%) 

Area  Test 
Score 

1 

10 

26  hrs. 

80 

78 

1+ 

12 

21  hrs. 

85 

80 

C .  Comparisons  of  CAI  with  Group  Instruction 

1 .  Experimental  Design  and  CAI  Modules  Used 

The  five  CAI  Modules  developed  during  this  project  were  used  in 
operational  tests  which  compared  CAI  with  standard  classroom  instruction. 

Each  module  was  evaluated  separately.  The  CAI  modules  were  AC  Theory, 
Inductance,  Capacitance,  Direct  Current,  and  Series  Circuits  and  Resonance. 
For  details  of  the  operational  tests  of  AC  Theory,  Inductance  and  Capacitance 
see  Ford  and  Slough  (1970),  Hurlock  (l97l)>  and  Hurlock  (1972). 

Students  who  received  CAI  training  were  randomly  selected  from 
classes  ready  to  receive  instruction  over  the  same  training  objectives 
included  in  the  CAI  material.  In  other  words,  all  students  used  in  CAI 
testing  were  taken  from  regular  classrooms,  and  it  was  the  responsibility  of 
the  project  to  see  that  these  students  received  instruction  over  the  same 
training  objectives  they  would  have  received  if  they  had  remained  in  class. 

At  the  end  of  CAI  training,  CAI  students  had  to  be  able  to  return  to  the 
BE/E  School  and  resume/  normal  classroom  instruction  with  their  regular 
class.  This  was  the  situation  which  existed  from  the  very  first  CAI 
module  tryouts  through  the  operational  test  phase. 

The  basic  experimental  design  employed  to  compare  CAI  with  group 
instruction  (classroom  training)  was  a  simple  two-group,  experimental/ 


control  experiment: 

Group 

Training 

Supplemental  Test 

School  Exam 

Experimental 

CAI 

Yes 

Yes 

Control 

Classroom 

Yes 

Yes 

CAI  students  were  treated  exactly  like  control,  classroom  students 
except  for  the  training  mode,  CAI.  Both  groups  received  both  the  CAI 
Supplemental  Test  and  the  School  Area  Examination  at  the  end  of  training. 
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2 .  Results  of  Comparisons  of  CAI  with  Group  Instruction 


The  results  of  comparisons  of  the  five  CAI  modules  with  standard 
classroom  instruction  are  summarized  in  Table  9-  The  summary  includes 
data  expressed  as  means  for  CAI  and  class  on  training  time  and  test  perfor¬ 
mance  scores  as  well  as  attitude  information  obtained  from  CAI  students  at 
the  end  of  CAI  training.  CAI  Inductance  and  Capacitance  Module  data  are 
partially  combined  because  they  comprised  the  School  Area  Five  curriculum 
and  were  evaluated  together  during  the  operational  test  of  the  capacitance 
module. 


TABLE  9 

Comparisons  of  CAI  and  Class  Performance 


Comparisons 

ML 

Theory 

Inductance /Capacitance 

DC 

Oti  X  C  o 

Circuits 

N 

CAI 

sf( 

6h 

50 

50 

Class 

200 

6  it 

20 

6U 

Training 

CAI 

9-5* 

8.1* 

7.6 

5.5* 

17.7* 

Time  (hrs) 

Class 

15.6 

17.0 

17.0 

10.3 

3Uj0 

Supplemental 

CAI 

92* 

85* 

90* 

76* 

86* 

Test  (%) 

Class 

82 

80 

78 

58 

79 

School 

CAI 

9!*# 

85* 

88* 

82 

Examination  (#) 

Class 

87 

80 

85 

80 

%  CAI  Preferred 

70. U 

70. It 

82.7 

78.2 

CAI  Rating 

it. 6 

it. It 

It. 3 

It.l 

*Stat istically  significant  compared  to  class  instruction. 


The  CAI  modules  showed  39  to  5k%  savings  in  comparison  to  correspond¬ 
ing  classroom  training  time.  These  differences  were  statistically  significant 
for  all  five  modules  at  the  .001  level. 

The  CAI  students  scored  significantly  higher  than  classroom  students 
on  all  module  supplemental  tests  at  the  .001  level.  Statistical  differences 
at  the  .001  level  were  also  found  between  groups  on  the  AC  Theory  and  on  the 
Inductance/Capacitance  School  Examinations  and  at  the  .05  level  on  the  DC 
School  Examination.  The  CAI  and  Class  students  did  not  differ  in  their 
performance  on  the  Series  Circuits  School  Examination. 

Attitude  questionnaire  answered  by  CAI  students  at  the  end  of  their 
CAI  training  indicated  that  if  given  a  choice,  they  would  prefer  over  70$ 
of  their  entire  BE/E  School  training  via  CAI.  On  a  five  point  scale,  where 
1  =  very  poor,  3  =  average  and  5  =  outstanding,  CAI  students  rated  CAI 
training  with  an  average  of  k.k  points. 
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D.  Effects  of  Lons  Term  CAT 

The  purpose  of  this  research  was  to  determine  if  there  is  any  decrement 
in  student  performance  or  motivation  as  a  result  of  extended  experience 
with  CAI •  A  comparison  was  made  between  students  who  took  a  single  CAI 
module  without  any  prior  CAI  and  students  who  had  had  prior  experience  on 
one  to  four  CAI  modules.  Data  on  the  performance  of  students  taking  only 
a  single  module  of  CAI  had  already  been  collected  in  the  operational  tests 
previously  described.  An  additional  50  students  were  given  all  five  modules 
consecutively.  These  students  were  randomly  selected  in  the  same  manner  as 
in  the  short  term  single  module  studies.  Analysis  of  aptitude  scores 
indicated  that  the  short  term  and  long  term  groups  were  equal  in  ability. 

Long  term  CAI  was  found  to  be  beneficial  rather  than  detrimental  to 
student  performance.  The  top  part  of  Table  10  shows  the  experimental  design. 
For  AC  Theory,  Inductance  and  the  first  half  of  Series  Circuits  (SC  1-10 ), 
the  short  term  CAI  students  had  zero  hours  of  prior  CAI.  During  training 
on  these  modules,  lower  half  of  Table  10,  the  long  term  CAI  students  obtained 
time  savings  over  the  short  term  CAI  students  ranging  from  7  to  23$. 


TABLE  10 

Mean  Time  Savings  Produced  by  Extended 
Experience  with  CAI 


Module : 

ACT 

IND 

SC  1-10 

CAP 

SC  11-20 

Prior  CAI  (hrs.) 

Short  Term  CAI: 

0 

0 

0 

8.1 

8.5 

Long  Term  CAI: 

7-4 

lU.7 

29-6 

22.2 

37-3 

Training  Time  (hrs,) 

Short  Term  CAI: 

9-5 

8.1 

7.3 

7-6 

5-2 

Long  Term  CAI: 

7-3 

7-5 

6.4 

7.4 

5-1 

%  Time  Saved 

23 

7 

12 

3 

2 

For  short  term  CAI,  students  taking  Capacitance  or  the  last  half  of 
Series  Circuits  (SC  11-20)  had  already  had  about  eight  hours  of  CAI.  These 
students  did  not  differ  from  the  long  term  students  in  training  time,  as 
the  right  side  of  Table  10  shows  ( 2%  and  3%>  difference).  Apparently, 
students  exhibit  a  small  amount  of  "learning  how  to  learn"  in  CAI  and  this 
process  takes  no  more  than  about  eight  hours  of  experience. 

Table  11  shows  that  both  groups  were  practically  the  same  in  module  test 
performance  and  student  ratings  of  CAI.  However,  long  term  CAI  resulted  in 
a  measurable  increase  in  the  preference  of  students  for  CAI  rather  than 
standard  instruction.  The  superiority  of  CAI  is  not  due  to  novelty. 
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TABLE  11 


Mean  Posttest  Performance  and  Student  Ratings 
in  Short  Term  and  Long  Term  CAT 


Area 

Suppl . 

CAI 

%  CAI 

Tests 

Tests 

Rating* 

Preferred 

Short  Term 

86 

88 

U.5 

71 

Long  Term 

86 

88 

k.2 

83 

*  1  =  very  poor;  5  =  outstanding 


E.  Paired  Students  Trained  by  CAI 

An  experiment  was  conducted  to  determine  if  students  working  in  pairs 
at  a  CAI  terminal  were  as  effective  as  students  working  individually 
(Hurlock  &  Hurlock,  1972).  The  problem  has  potential  implications  for  cost 
feasibility  of  CAI  as  well  as  for  research  on  learning  and  instruction.  The 
CAI  Inductance  and  Capacitance  Modules  were  used  for  the  experiment. 

Students  were  randomly  assigned  to  the  experimental  (pairs)  and  control 
(individual)  groups.  During  training  the  paired  students  exchanged  positions 
from  lesson  to  lesson  in  the  role  of  using  the  keyboard  to  answer  questions. 
All  students  in  the  experiment  took  the  two  module  tests  and  the  Area 
Examination  individually . 

No  significant  differences  were  found  between  students  trained  in  pairs 
(N  =  50)  and  students  trained  alone  (N  =  25)  in  performance  on  two  major 
tests  and  a  comprehensive  examination  or  in  training  time.  Mean  percent 
correct  scores  on  the  CAI  Inductance  and  Capacitance  Module  Tests  and  on 
the  School  Area  Examination  for  pairs  were  86.0,  90.4,  and  83.9 5  respectively. 
Mean  scores  for  individuals  were  89*15  92.6,  and  84.8,  respectively.  Mean 
training  time  was  869  minutes  for  pairs  and  820  minutes  for  individuals. 

The  data  strongly  supported  the  hypothesis  that  students  can  be  trained 
in  pairs  at  a  CAI  terminal  without  degradation  in  performance  or  in  training 
time  over  that  expected  for  students  trained  alone.  In  conclusion,  it 
appears  that  paired  student  training  is  a  potentially  feasible  method  to 
reduce  the  cost  per  terminal  hour  of  CAI,  where  CAI  course  materials  are 
basically  fixed  sequenced  and  where  students  are  paired  on  the  basis  of 
learning  rate  and  aptitude. 

F.  Evaluation  of  Branching  Technology 

In  this  section,  brief  summaries  are  given  of  experimental  evaluations 
of  six  types  of  branching  technology  previously  described  in  Section  II. 

Most  of  these  techniques  were  developed  and  incorporated  into  CAI  materials 
for  the  operational  tests,  including  the  long  term  CAI  test.  Two  of  the 
techniques  were  tested  in  separate  experiments  at  the  conclusion  of  the 
operational  tests. 


1 .  Remedial  Branching 

This  study  evaluated  the  effectiveness  of  the  CAI  branched  remedial 
pretraining  previously  described  in  Section  II. B. 4.  The  pretraining  was 
given  just  before  the  CAI  module  on  DC  series-parallel  circuits  and  provided 
remediation  for  students  coming  from  class  instruction  on  simpler  circuits. 
The  pretraining  was  needed  by  most  students  because  analysis  of  student 
tryout  data  showed  that  the  ability  to  analyze  simple  series  and  parallel 
circuits  was  a  necessary  prerequisite  to  training  on  more  complex  series- 
parallel  circuits. 

The  remediation  was  designed  to  be  maximally  adaptive  to  individual 
differences  in  knowledge  of  prerequisites.  There  were  three  remediation 
lessons.  In  each  lesson,  the  student  could  bypass  training  on  any  of  the 
objectives.  However,  if  the  student  missed  an  item  on  the  lesson  test  and 
had  skipped  the  pertinent  training,  he  was  immediately  branched  to  that 
training  before  continuing  with  the  test.  (Refer  to  Figure  4.) 

A  control  group  of  15  students  took  the  CAI  DC  Module  without  any 
remediation.  The  experimental  group  (N  =  50 )  consisted  of  students  used  in 
the  Long  Term  CAI  study.  They  were  given  the  CAI  remedial  pretraining  before 
taking  the  CAI  DC  Module.  Results  are  shown  in  Table  12. 


TABLE  12 

Remediation  Time  and  Module  Test  Performance 


Remediation 

Time 

(min .  ) 

Mean  Module 

Scores 

Lessons 

Tests 

Lesson 

Criterion 

School 

Mean 

Range 

Mean 

Range 

Tests 

Test 

Exam 

No  Remediation 

— 

— 

— 

— 

69 

58 

78 

Remediation 

57 

2-198 

58 

19-134 

78 

76 

88 

The  remediation  appeared  to  be  very  adaptive  to  individual  differences. 
Total  remediation  time  ranged  from  2  to  198  minutes  on  the  lessons  and  19  to 
13*+  minutes  on  the  lesson  tests.  The  average  student  required  less  than  two 
hours  (57  +  58  minutes)  for  the  entire  pretraining.  This  pretraining 
provided  remediation  for  about  22  hours  of  previous  class  instruction. 

Test  performance  on  the  CAI  DC  Module  is  shown  in  the  right  half  of 
Table  12.  Students  who  had  received  remediation  scored  significantly  higher 
on  all  three  tests  than  students  who  had  not  received  remediation.  The 
difference  was  particularly  large  for  the  CAI  criterion  test  for  the  series- 
parallel  training.  This  was  a  very  difficult  test  -  both  CAI  students  who 
did  not  receive  remediation  and  students  who  took  their  entire  training  in 
class  averaged  58%. 


42 


2. 


Pretest  Branching  Designs 


The  instructional  strategies  for  these  two  designs.  Pretest  Design 
I,  "Skip  Ahead,"  and  Pretest  Design  II,  "Minimal  Training,"  were  discussed 
in  Section  II. B. 5.  Two  lessons  in  the  CAI  Capacitance  Module  were  con¬ 
structed  to  serve  as  evaluation  vehicles.  Lesson  3  (CAP  6)  incorporated 
the  "skip  ahead"  strategy,  and  Lesson  6  (CAP  6)  used  the  "minimal  training" 
strategy.  Time  and  manpower  restrictions  prevented  an  experimental  comparison 
with  fixed  sequence  training  controls  using  the  same  lessons.  It  was  there¬ 
fore  necessary  to  evaluate  the  pretest  designs  descriptively  against  two 
other  lessons,  CAP  5  and  CAP  7>  which  were  developed  as  fixed  sequence  train¬ 
ing  lessons. 

During  the  operational  test  of  the  CAI  Capacitance  Module,  frame-by- 
frame  computer  performance  data  were  recorded  for  each  of  6h  experimental 
students  as  they  proceeded  through  the  lessons.  Lesson  training  time, 
beginning  with  the  first  lesson  frame  and  ending  with  the  last  lesson  frame, 
and  lesson  test  scores  ( %  correct)  were  also  recorded.  Performance  data  on 
each  lesson  were  sorted  and  listed  for  each  student.  Student  trails, 
student-by- frame,  were  analyzed  to  obtain  a  count  of  total  frames  encountered. 
A  frame  was  counted  only  once,  i.e.,  retrys  and  light  pen  mismatches  were 
not  counted. 

Lesson  test  scores  were  high  on  both  the  experimental  and  the  control 
lessons.  Median  scores  on  the  lessons  ranged  between  88  and  96$.  All  four 
lessons  appeared  to  be  instruct ionally  effective,  if  not  equal.  These  data 
were  interpreted  as  being  supportive  justification  for  making  further 
comparisons  between  the  experimental  and  the  control  lessons. 

Amount  of  training  received  was  determined  by  comparing  a  count  of 
the  total  number  of  frames  encountered  by  each  student  in  each  lesson.  The 
hypothesis  that  pretest  branching  designed  lessons  would  permit  large 
individual  variations  in  the  amount  of  training  given  to  each  student  was 
positively  supported.  This  was  especially  shown  in  the  measures  of  vari¬ 
ability  such  as  range  (1*6-101  and  13-56),  range  ratio  (2.1  and  1+.3)  and 
coefficient  of  variation  (15*1+  and  U2.2)  for  CAP  3  and  CAP  6,  respectively. 

By  definition,  there  was  no  variability  in  the  fixed  sequence  lessons  for 
amount  of  training. 

Training  time  to  complete  instruction  in  each  lesson  is  presented 
in  Table  13.  Although  the  amount  of  training  (mean  number  of  frames) 
received  in  CAP  5  and  CAP  7>  fixed  sequence  lessons,  (U5  and  60  frames)  fell 
between  that  of  the  two  pretest  lessons  (73  and  23  frames),  students  took 
two  to  four  times  longer  to  complete  the  control  lessons.  Pretest  lessons 
appeared  to  allow  for  large  individual  variations  in  training  time,  especially 
in  the  direction  of  completing  training  rapidly. 

Range  ratios  (ratio  between  the  longest  time  and  the  shortest  time) 
and  coefficients  of  variation  for  the  pretest  lessons  were  much  larger  than 
those  for  the  control  lessons.  Figure  l6  shows  that  pretest  training  lessons 
produced  positively  skewed  completion  times,  while  fixed-sequence  lessons 
resulted  in  training  time  approximating  a  normal  distribution. 


h3 


TABLE  13 


Training  Time  on  Pretest  and 
Fixed-Sequence  Lessons 


Pretest 

CAP- 3  CAP-6 

Fixed 
CAP- 5 

Sequence 

CAP-T 

Median  (minutes) 

20.2 

13.9 

56.5 

46.7 

SD 

7.6 

9-9 

9-5 

9-9 

Range 

12-1+7 

5-60 

37-83 

31-70 

Range  Ratio 

4.0 

12.0 

2.2 

2.3 

Coef.  Var. 

34.2 

59-5 

16.8 

20.7 

NUMBER 

OF 

STUDENTS 


Figure  id.  Training  time  distribution  of  Pretest  and 
Fixed  Sequence  lessons 
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The  evaluation  of  "skip  ahead"  and  "minimal  training"  pretest  designs 
appeared  to  demonstrate  that  adaptive  training  allowed  each  student  to 
receive  the  amount  of  instruction  that  he  needed  on  each  training  objective. 
As  the  number  of  training  frames  encountered  by  students  tended  to  be 
positively  skewed  or  normal,  the  importance  of  pretest  or  individually 
adaptive  branching  is  supported.  It  indicates  that  the  majority  of  students 
need  only  minimal  or  selective  training.  The  pretest  designed  lessons 
allowed  the  fast  learner  to  take  50  to  75#  less  instruction  than  the  naive 
or  slow  learner.  Training  time  differences  between  students  going  through 
the  adaptive  lessons  were  very  pronounced.  Pretest  designed  lessons  allowed 
fast  learners  to  proceed  through  the  lessons  h  to  12  times  more  quickly  than 
slow  learners.  On  the  other  hand,  in  fixed  sequence  CAI  lessons  fast 
learners  could  complete  lessons  no  more  than  twice  as  quickly  as  slower 
learners. 

3-  Multiple  Branching 

Much  of  CAI  has  followed  a  fixed  sequence  structure  with  extensive 
internal  branching  within  frames  (retries  and  hint  frames)  but  with  limited 
opportunities  for  external  branching  involving  whole  sequences  of  frames. 

The  present  study  (Slough,  et  al.  1972)  compared  fixed  sequence  lessons 
with  branching  lessons  which  incorporated  a  variety  of  branching  applica¬ 
tions  under  the  control  of  the  program  or  the  student  or  both. 

Lessons  C  and  E  of  the  CAI  Series  Circuits  and  Resonance  Module  were 
used  for  this  research.  In  their  original  form,  the  lessons  were  basically 
fixed  sequence  lessons  with  little  opportunity  for  external  branching.  The 
revised  version  of  lesson  X)  provided  multiple  opportunities  for  branching 
in  remediation,  instruction,  practice,  and  review.  The  branching  version 
of  lesson  E  provided  preliminary  instruction  in  the  student’s  Study  Guide 
and  emphasized  the  use  of  dual  control  practice  for  the  CAI  portion  of  the 
training.  Flow  charts  of  the  branching  versions  of  lessons  C  and  E  have 
previously  been  shown  in  Figure  7  and  Figure  8,  Section  II. B. 

The  first  branching  versions  produced  a  very  large  time  savings  over 
fixed  sequence  instruction.  However,  there  was  also  a  loss  in  test  perfor¬ 
mance  so  the  lessons  were  revised.  The  final  branch  versions  provided 
increased  scope  for  branching  and  optional  formal  review  at  the  end  of  the 
lessons . 


Table  ih  compares  performance  on  the  fixed  sequence  version  with 
performance  on  the  final  branching  version.  Branching  produced  a  significant 
(p<.0l)  and  very  substantial  time  savings  in  both  lessons.  Furthermore, 
branching  was  equally  effective  for  students  of  high  and  low  aptitude. 

Scatter  plots  and  regression  analysis  indicated  that  average  time  savings 
were  the  same  at  all  aptitude  levels  sampled. 

Although  test  scores  decreased  slightly,  performance  on  the  branch¬ 
ing  versions  was  quite  satisfactory  (9^  and  93 l) • 
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TABLE  111 


Mean  Training  Times  and  Test  Scores  for 
Fixed  Sequence  and  Branched  Instruction 


Measure : 
Lesson: 


Time  (min.) 
C  E 


Score  (%) 
C  E 


Fixed  Sequence:  77  8l  98  94 

Branching:  33  57  94  93 


A  comparison  of  this  experiment  with  previous  research  suggests 
that  the  following  factors  are  particularly  critical  in  designing  branched 
instruction:  (l)  The  usefulness  of  optional  bypassing  of  initial  instruc¬ 

tion  depends  on  the  extent  to  which  the  student  is  provided  with  exact 
knowledge  of  the  course  objectives;  (2)  branching  programs  are  most  useful 
when  there  is  some  prior  student  attainment  of  these  objectives,  or  if  there 
is  provision  for  pretraining  or  concurrent  training  which  produces  a  range 
of  attainment;  (3)  f,dual-controlM  practice  provides  a  useful  balance  between 
complete  program  control  and  complete  student  option;  (4)  availability  of 
formal  review  is  very  important  in  providing  sufficient  retention. 

4 .  Branching  in  Drill  and  Practice 

This  study  compared  program  control  and  dual  control  methods  for 
determining  the  amount  of  practice  to  be  given  to  each  student.  Lesson  15 
of  the  Series  Circuits  and  Resonance  Module  was  used  as  the  research  vehicle. 
This  lesson  was  a  drill  and  practice  lesson  on  analysis  of  LCR  circuits. 
Training  on  each  objective  consisted  of  practice  on  the  two  components  of 
the  objective  (e.g.,  find  phase  angle  and  impedance)  followed  by  a  criterion 
problem  which  combined  both  components. 

Two  versions  of  the  lesson  were  prepared.  In  the  program  control 
version,  practice  was  continued  until  the  student  had  done  two  problems 
correctly  on  the  first  try.  The  student  then  did  the  criterion  problem 
and  went  on  to  the  next  objective.  In  the  dual  control  version,  the 
practice  was  completely  optional.  However,  if  the  student  missed  the 
criterion  problem  and  had  skipped  the  practice,  he  was  required  to  go  back 
and  do  the  practice.  Practice  continued  until  he  did  one  problem  correctly. 

Results  are  shown  in  Table  15.  Dual  control  practice  required  much 
less  time  than  program  controlled  practice,  and  as  might  be  expected,  the 
difference  was  statistically  significant  (p<.00l).  Although  test  perfor¬ 
mance  was  higher  in  the  dual  control  version,  it  was  not  significantly 
different  from  the  program  control  version. 

These  findings,  combined  with  those  from  the  previous  section, 
clearly  indicate  that  strict  program  control  of  drill  and  practice  is  less 
efficient  than  dual  control  procedures  where  the  student  is  allowed  to 
enter  into  the  decision  process. 


46 


TABLE  15 


Comparison  of  Program  Control  and  Dual 
Control  Branching  in  Drill  and  Practice 


Measure:  Training  Time  (min.)  Test  Score  (%) 


Program  Control:  69  87 

Dual  Control :  39  95 


5.  Student  Control  Versus  Program  Control  in  Training 

The  goals  of  this  study  were  to  compare  two  instructional  strategies 
for  individualizing  CAI  training  materials,  and  to  evaluate  the  effect  of 
providing  a  lesson  narrative  before  training. 

Two  types  of  adaptive  instructional  strategies  were  compared:  (l) 
the  student  selected  his  own  training,  and  (2)  the  course  program  controlled 
training  for  the  student  based  on  his  pretest  results.  The  influence  of 
having  the  student  read  a  narrative  overview  of  training  content  before 
CAI  instruction  on  each  lesson  was  also  examined.  The  course  material 
was  a  modified  version  of  the  previously  developed  CAI  AC  Series  Circuits 
and  Resonance  Module. 

Ninety-six  students  from  BE/E  School  were  divided  into  four  different 
instructional  strategy  groups  for  taking  the  11-lesson  CAI  Module.  One 
training  strategy  allowed  the  student  to  select  his  training  from  an  index 
of  descriptive  lesson  objectives.  A  second  training  strategy  pretested  the 
student  immediately  before  each  lesson  objective  and  branched  him  to 
appropriate  training  sequences  on  the  basis  of  his  test  results.  Each  of 
these  two  strategies  were  used  with  and  without  a  narrative  presentation 
before  each  lesson  to  make  the  third  and  fourth  experimental  training 
conditions . 

At  the  end  of  training  and  after  completing  an  attitude  question¬ 
naire  about  their  CAI  training  experience,  all  four  groups  took  the  School 
Area  Examination  and  a  Supplementary  Test  comprised  of  school  training 
objectives  not  represented  on  the  examination. 

No  significant  differences  were  found  between  the  four  experimental 
conditions  in  test  performance  or  training  time  measures.  Questionnaire 
data  indicated  that  students  who  selected  their  own  training  maintained  a 
significantly  more  favorable  attitude  toward  CAI.  In  addition,  students 
who  had  a  pre-training  narrative  available  to  them  felt  that  it  was  a 
valuable  aid. 

The  best  indicators  of  CAI  training  success  were  scores  on  previous 
school  examinations  and  prior  time  spent  in  the  BE/E  School’s  individualized 
training  curriculum.  Performance  on  the  CAI  Module  was  not  significantly 
related  to  General  Classification  Test  scores  or  to  two  other  aptitude 
measures. 


6.  Effects  of  Student  Control  in  Remediation 

This  study  questioned  whether  student  choice  might  prove  to  be  the 
better  way  to  select  remediation  training  since  the  student  would  at  that 
point  have  a  better  understanding  of  the  lesson  objectives.  Course  material 
was  a  modified  version  of  the  previously  developed  CAI  AC  Series  Circuits 
and  Resonance  Module. 

Two  modes  of  lesson  presentation  were  used:  student  option  and 
pretest  program  control.  All  students  took  Lessons  1  and  2  in  the  same 
manner.  For  the  remaining  nine  lessons,  half  of  the  students  took  training 
in  odd-numbered  lessons  in  the  pretest  mode,  and  training  in  even-numbered 
lessons  in  the  student  option  mode.  The  sequence  was  reversed  for  the  other 
half  of  the  students  to  counterbalance  the  design. 

Students  were  also  assigned  to  one  of  three  remediation  modes: 
student  option,  program  control,  and  test /retest.  Remediation  followed 
the  lesson  test  if  the  student  failed  to  pass  a  criterion  of  roughly  Q0% 
correct.  The  student  option  groups  could  take  as  much  or  as  little  of  the 
scheduled  remediation  training  as  they  chose.  The  program  control  groups 
took  all  scheduled  remediation.  The  test/retest  group  received  and  went 
directly  to  the  lesson  remediation  test.  The  results  indicated  no  dif¬ 
ference  in  performance  as  a  result  of  either  lesson  treatment  or  remediation 
treatment.  This  suggests  (l)  that  student  choice  is  at  least  as  effective 
as  program  control  as  a  training  strategy,  and  (2)  that  remediation  which 
occurs  immediately  after  a  posttest  is  ineffective.  Since  student  choice 
was  preferred  by  students,  and  since  it  eliminates  many  problems  in  the 
preparation  of  training  materials,  it  promises  to  be  a  superior  training 
strategy  for  CAI  lessons. 

IV.  Evaluation  of  IBM  1300  System 

An  evaluation  was  made  of  both  the  technical  and  economic  feasibility  of 
the  IBM  1500  system.  Technical  feasibility  included  general  training  effec¬ 
tiveness,  hardware  and  software  limitations,  and  operational  limitations. 

The  evaluation  was  derived  from  empirical  data  and  experience  in  developing 
CAI  for  the  application  in  basic  electronics  which  was  described  in  earlier 
sections  of  this  report.  The  evaluation  of  any  CAI  system  must  take  into 
account  many  factors,  and  these  must  be  specified  and  evaluated  in  terms  of 
the  specific  application.  It  is  not  intended  that  this  evaluation  be 
generalized  to  other  applications  where  some  of  the  factors  which  were  found 
to  be  critical  in  this  application  may  have  substantially  different  levels 
of  criticality. 

A.  Technical  Feasibility 

1 .  General  Training  Effectiveness 

a.  Achievement  of  students.  One  of  the  first  questions  which  must 
be  answered  is  whether  the  1500  system  can  be  used  to  provide  effective 
training.  Based  upon  the  results  of  the  five  CAI  module  evaluations  as 


48 


— - 


well  as  the  long  term  CAI  study,  the  answer  to  that  question  is  that  the 
training  is  very  effective*  CAI  students  scored  consistently  higher  than 
students  given  conventional  instruction.  (See  Section  III  for  details.) 

b.  Efficiency  of  training.  Not  only  should  training  be  effective 
but  it  should  be  efficient.  Time  comparisons  of  CAI  with  classroom  training 
on  comparable  sets  of  training  objectives  showed  that  CAI  training  required 
about  h5%  less  time.  (See  Section  III.) 

c.  Student  reactions.  Student  reactions  to  the  CAI  programs  were 
predominantly  favorable  with  but  a  few  exceptions.  They  preferred  to  take 
the  bulk  of  their  training  via  CAI.  This  was  true  whether  they  had  experi¬ 
ence  with  one  CAI  module  or  with  all  five  modules.  (See  Section  III.) 

d.  Output  of  information  on  student  progress.  While  the  1500 
system  could  not  perform  detailed  analysis  of  student  performance  records 
simultaneously  with  providing  CAI  at  the  terminals,  several  student  perfor¬ 
mance  summaries  were  developed  for  on-line  use.  These  proctor  messages 
were  especially  useful  to  instructors  for  spotting  student  weaknesses  and 
for  managing  the  progression  of  individual  students  through  the  course. 

These  messages  also  provided  the  lesson  designers  with  a  "quick  look"  at 
the  effectiveness  of  lessons.  In  addition,  a  variety  of  off-line  programs 
were  developed  in  this  project  to  provide  more  detailed  course  analysis. 
These  made  the  1500  system  a  very  effective  instrument  for  course  evaluation 
and  revision.  (See  Sections  II. C.,  II. D.,  and  III.B. ) 

It  may  be  concluded  that  the  1500  system  provides  very  effective 

training. 

2.  Hardware  and  Software  Limitations 


There  are  a  number  of  limitations  of  the  1500  system  which  make  it 
unsatisfactory  as  a  training  system  for  this  application.  Some  of  these 
limitations  may  not  be  as  serious  in  certain  other  education  or  training 
applications  or  in  research  and  development  applications. 

a.  Coding.  The  language  in  which  the  CAI  lessons  were  written  is 
an  authoring  language  called  COURSEWRITER  II.  Writing  code  for  lessons  in 
this  language  is  a  long  and  tedious  task  except  for  lessons  with  very  simple 
logic.  It  is  not  likely  that  instructional  personnel  will  be  able  to  code 
in  this  language  without  substantial  training  and  considerable  time  being 
made  available.  In  this  project  most  of  the  lesson  material  was  developed 
by  research  and  instructional  personnel  using  special  lesson  planning  guides. 
Then  college  students  trained  in  the  use  of  COURSEWRITER  coded  and  debugged 
the  lessons  at  CAI  terminals. 

b.  Programming  of  advanced  instructional  logics.  Most  advanced 
CAI  techniques,  such  as  student  interaction  with  graphic  simulation  models, 
require  complex  instructional  logics.  While  COURSEWRITER  II  can  be  used  to 
implement  some  of  these,  the  programming  is  cumbersome  and  results  in  large 
amounts  of  coding.  With  the  limited  amount  of  on-line  direct  access  storage 
capacity  for  lessons,  this  limitation  must  be  given  even  greater  weight. 
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c.  Graphic  di splays .  Because  author  developed  graphics  are  stored 
in  core  memory  and  because  of  limitations  of  core  memory  (32K  words),  there 
are  severe  limitations  on  the  use  of  graphics  with  the  1500  system.  Many 
advanced  CAI  techniques  have  a  heavy  requirement  for  graphics. 

d.  Amount  of  lesson  material  stored  on-line.  In  the  1500  system 
the  maximum  amount  of  storage  on  disk  drives  for  lesson  material  is  four 
million  characters.  The  five  CAI  modules  in  basic  electronics  require 
slightly  more  than  four  million  characters  of  storage.  If  all  lesson 
materials  are  not  accessible  at  one  time,  serious  problems  of  scheduling 
and  utilization  will  develop.  Thus,  when  CAI  lessons  are  designed  as  in 
this  project,  training  programs  in  excess  of  about  five  weeks  in  length 
would  require  more  than  one  1500  system  to  meet  the  requirement  of  having 
all  lessons  on-line  simultaneously. 

e.  System  response  time.  There  were  a  few  special  conditions  under 
which  the  response  time  of  the  system  degraded  seriously.  One  set  of  condi¬ 
tions  involved  the  updating  of  graphics  in  a  number  of  lessons  which  were 
being  used  at  the  same  time.  Under  these  circumstances,  system  response 
time  degraded  to  an  average  of  from  6  to  10  seconds.  On  a  few  occasions 
response  times  as  long  as  30  seconds  were  observed.  These  were  special 
conditions  encountered  in  only  a  few  lessons.  For  the  most  part  system 
response  times  were  under  five  seconds,  usually  less  than  one  second,  as 
required  by  the  specifications. 

f.  Audio  capability.  The  audio  units  had  several  deficiencies 
which  made  them  unsuited  for  training  purposes.  First  the  search  time  in 
finding  a  message  on  the  audio  tape  was  frequently  long  enough  to  disturb 
the  training  sequence  within  a  lesson.  Maximum  search  time  was  six  minutes, 
but  search  times  of  several  minutes  occurred  under  branching  conditions. 
Second,  if  the  program  failed  to  find  the  required  message,  the  station  was 
signed  off  and  the  intervention  of  a  proctor  was  needed  with  the  consequent 
interruption  of  a  training  sequence.  Third,  the  procedure  for  entering 
audio  message  addresses  into  C0URSEWRITER  programs  was  cumbersome  and  time 
consuming  and  required  special  training.  These  problems  were  so  severe  that 
course  authors  tended  to  use  less  audio  in  lessons  near  the  end  of  the 
project . 

3.  Operational  Limitations 

The  1500  system  has  a  serious  operational  limitation  with  respect 
to  its  employment  in  many  Navy  training  applications.  The  system  has  a 
limit  of  32  terminals.  Thus,  for  the  application  to  basic  electronics  many 
more  terminals  would  be  required,  since  the  daily  on  board  count  is  between 
300  to  500  students.  Such  an  expansion  would  require  the  acquisition  of 
additional  1500  systems.  This  solution  would  be  very  costly  and  would 
increase  the  operational  support  requirements  and  the  complexity  of  managing 
the  system.  The  32  terminal  limitation  not  only  imposes  severe  operational 
restrictions  but  also  adds  to  the  inherently  high  cost  per  terminal  output 
of  the  system.  Computer  equipment  such  as  central  processing  unit,  disk 
and  tape  drives,  line  printer,  card  reader,  and  multiplexing  equipment  may 
be  considered  as  computer  overhead  costs  which  must  be  prorated  across  the 
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number  of  terminals  served.  Thus,  a  system  with  hundreds  or  thousands  of 
terminals  will  generally  have  a  much  lower  prorated  cost  of  computer  equip¬ 
ment  than  the  1500  system,  even  though  the  computer  equipment  in  such  a 
system  may  cost  up  to  10  or  20  times  as  much. 

B.  Economic  Feasibility 

Economic  feasibility  refers  to  the  cost  of  operating  a  CAI  system  as 
compared  to  the  cost  of  an  alternative  program.  When  compared  to  the  cost 
and  effectiveness  of  the  classroom  training  program,  the  IBM  1500  system 
can  probably  achieve  sufficient  cost  savings  through  reductions  in  training 
time  to  pay  for  itself.  However,  the  BE/E  School  has  been  converted  to  an 
individualized  learning  system,  and  it  is  reported  that  average  training 
time  has  been  reduced  to  about  21  training  days  in  contrast  to  the  former 
six  week  curriculum.  Thus,  the  difference  in  training  time  between  CAI 
and  the  individualized  system  is  much  smaller  than  between  CAI  and  the  class¬ 
room  program. 

It  is  not  useful  to  introduce  detailed  cost  figures  on  the  IBM  1500 
system  since  the  configuration  used  in  this  project  had  only  1 6  terminals. 

The  lease  cost  for  this  configuration  was  approximately  $144,000  per  year 
including  maintenance.  Costs  of  the  1500  system  on  a  per  terminal  hour 
basis  range  from  about  $2.00  to  $4.00  depending  upon  accounting  procedures 
and  whether  a  one  or  two  shift  operation  is  assumed.  The  lower  figure, 

$2.00  per  terminal  hour,  is  several  times  larger  than  the  target  figure  of 
between  $0.40  to  $0.80  per  terminal  hour  estimated  for  some  third  generation 
CAI  systems.  Finally,  although  no  announcement  has  been  made,  it  is  assumed 
that  IBM  Corporation  is  no  longer  devoting  resources  to  develop  or  enhance 
the  1500  system. 


V.  Related  Applications 

Since  the  results  of  evaluation  indicate  that  CAI  is  a  very  effective 
method  of  instruction,  the  question  arises  as  to  the  feasibility  of  ship¬ 
board  CAI  as  well  as  other  shipboard  computer  applications  to  training.  It 
must  be  emphasized  that  the  following  discussion  is  based  upon  an  extrapola¬ 
tion  of  experience  with  CAI  at  a  shore  based  facility  and  not  upon  actual 
experience  with  CAI  on  shipboard.  Work  is  being  conducted  on  this  problem 
at  this  Laboratory  under  ADO  P43-03X.14,  Application  of  Shipboard  Computers 
to  Training  and  Training  Management. 

A.  Computer  Based  Training  Aboard  Ship 

1 .  The  Problem 


The  emphasis  on  more  and  better  training  aboard  ship  has  been 
increasing  steadily.  The  major  reasons  are  the  costs  and  inconveniences 
of  sending  shipboard  personnel  to  shorebased  training  centers,  and  the 
basic  fact  that  training  ashore  provides  even  the  best  student  with  terminal 
behaviors  that  are  not  synonymous  with  job  entry  behaviors  upon  his  return 
or  assignment  to  a  ship.  The  increased  emphasis  on  better  shipboard  training 
has  naturally  led  to  consideration  of  utilizing  computers  aboard  ship  for 
instructional  and  training  management  purposes. 
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The  kinds  of  computer  systems  being  considered  fall  into  two  main 
categories:  first,  those  already  installed  for  operational,  non-training 

purposes,  and  secondly,  those  which  could  feasibly  be  added  to  shipboard 
inventory  to  satisfy  training  needs  and  requirements.  The  basic  training 
applications  that  may  be  considered  for  both  categories  are  fourfold:  (l) 
simulation,  (2)  CAI,  (3)  CMI,  and  (4)  training  administration/management. 

2.  Shipboard  Operational  Computer  Systems 

A  major  breakthrough  in  utilization  of  shipboard  operational  computer 
systems  for  training  purposes  came  with  the  advent  of  the  Navy  Tactical  Data 
System  (NTDS).  The  NTDS  system  incorporates  a  video  simulator  component 
which  generates  up  to  32  synthetic  targets  for  training  purposes.  Opinions 
vary  as  to  whether  or  not  this  training  capability  has  been  effectively 
exploited  for  utilization  on  many  NTDS  ships.  A  library  of  well  designed 
exercises  is  recommended  for  training  individual  NTDS  operators  and  the 
entire  Combat  Information  Center  (CIC)  team.  Admittedly,  the  training 
capability  of  the  video  simulator  is  limited  compared  to  large  scale  shore- 
based  simulators  like  the  Tactical  Advanced  Combat  Direction  and  Electronic 
Warfare  System  (TACDEW).  However,  the  usefulness  of  the  simulator  for 
shipboard  use  has  been  amply  demonstrated. 

More  recently,  a  pioneer  precedent  for  shipboard  CAI  with  the  NTDS 
system  was  PROF-E  (Programmed  Review  of  Operator  Functions-Elementary )  for 
the  ASW  function  aboard  a  few  ships.  A  later  development  was  L-TRAN  (Lesson 
Translator  Program)  which  may  be  described  as  an  AAW  counterpart  of  PROF-E. 
Various  constraints  inherent  in  PROF-E  show  greater  promise  of  resolution 
in  L-TRAN.  Work  on  L-TRAN  has  emphasized  feasibility  aspects  with  respect 
to  computer  programming  and  equipment,  and  much  needs  to  be  done  in  these 
two  areas . 

Prototype  training  packages  are  being  delivered  to  NTDS  ships  but 
continued  development  work  is  needed  in  the  areas  of  instructional  design 
and  preparation  and  validation  of  training  materials. 

Possibilities  for  CMI  aboard  ship  utilizing  presently  installed 
computer  systems  are  more  promising  than  those  for  simulation  and  CAI.  The 
basic  reasons  are  fewer  requirements  for  equipment,  programming,  and  on-line 
time.  The  problems  with  respect  to  lessonware  are  of  equal  or  greater 
magnitude  since  several  instructional  media  are  involved. 

Shipboard  computers  which  have  at  least  a  minimum  potential  for  CMI 
as  well  as  for  training  administration  and  management  are  the  AN/UYK-5  and 
the  CP789.  Both  would  be  used  in  the  batch  data  processing  mode.  There 
may  be  serious  problems  of  availability  of  the  CP789  when  underway.  The 
AN/UYK-7  holds  much  greater  potential  for  use  in  both  applications  as  well 
as  limited  applications  for  CAI.  Again  the  availability  of  the  system  is 
a  serious  question. 
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3.  Minicomputers  and  Other  Alternatives 


An  alternative  to  the  use  of  shipboard  computers  installed  for 
operational  purposes  is  the  installation  of  minicomputers  configured  for 
training  and  training  management  applications.  A  configuration  consisting 
of  the  following  would  probably  be  required  for  CAI  or  CMI  applications: 

CPU  with  32K  words  core  storage,  disk  units,  magnetic  tape  units,  card 
read/punch,  line  printer,  multiplexer,  typewriters  and  keyboard  cathode  ray 
tubes.  An  authoring  language  such  as  APL,  BASIC,  or  a  similar  one  would  be 
required.  Such  a  configuration  would  meet  or  exceed  the  requirements  for 
training  management  provided  that  the  data  file  requirements  were  not  too 
large.  The  minicomputer  would  have  many  advantages  for  these  applications, 
the  most  important  being  flexibility  of  scheduling.  A  major  disadvantage 
is  the  cost  of  the  system. 

A  second  alternative  is  to  use  terminals  aboard  ship  which  are 
connected  to  a  large,  distributed  terminal  computer  system  ashore.  Com¬ 
munication  would  probably  be  by  satellite.  There  are  many  problems  which 
must  be  solved  before  such  a  system  configuration  could  become  operational. 
Among  these  are  communication  problems,  development  of  libraries  of  courses 
(both  CAI  and  CMl),  and  utilization  and  training  management  aboard  ship. 

B.  Uses  of  Computer  Based  Instruction  in  a  Training  Program 

Experience  in  this  and  other  projects  suggests  three  main  alternatives 
in  the  use  of  computer  based  training  methods:  (l)  mainline  CAI,  (2)  CMI 
employing  remote  batch  terminals  or  interactive  terminals,  and  (3)  mixed 
mode  CAI /CMI. 

In  mainline  CAI,  a  major  portion  of  the  instruction  is  provided  at  an 
interactive  computer  terminal  and  the  instructional  functions  include 
presentation,  practice,  correction,  testing  and  remediation.  In  CMI  the 
instruction  is  provided  by  non-computer  methods.  Computer  functions  are 
employed  to  record  and  analyze  student  progress  tests,  provide  test  scores 
and  diagnosis  of  training  deficiencies  and  (optionally)  prescribe  training 
and  materials  for  practice  and  remediation.  In  mixed  mode  CAl/CMI,  non¬ 
computer  training  modules  are  under  sequence  and  control  of  CMI  functions. 

CAI  is  available  for  specialized  practice  and  remediation,  and  for  mainline 
instruction  for  selected  parts  of  the  training.  All  of  these  alternatives 
employ  individualized  instruction  methods,  that  is,  individually  administered, 
modularized  training  materials.  The  designs  may  also  be  applicable  to  small 
tutorial  groups. 

While  each  of  these  alternatives  might  be  most  feasible  for  a  given 
application,  several  advantages  make  the  mixed  mode  CAl/CMI  the  most 
attractive.  First,  it  allows  for  incremental  development  of  computer  based 
training  systems;  all  development  problems  need  not  be  solved  before  the 
computer  based  training  operations  can  start.  Second,  computer  based 
support  is  provided  for  individualized  curriculum  development  and  testing. 
Third,  this  design  provides  a  highly  controlled  learning  environment  when 
it  is  needed.  Fourth,  cost  effective  off-line  training  can  be  used  for 
large  portions  of  training.  Fifth,  such  a  method  provides  a  facility  for 
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development  and  testing  of  training  modules.  Sixth,  the  modularized, 
computer  based  features  of  the  training  system  permit  the  entire  training 
system  to  be  moved  and  operated  effectively  at  a  site  remote  from  a  formal 
school  without  a  complete  replication  of  the  school.  For  example,  an  entire 
curriculum  can  be  installed  in  trailers  or  vans  and  transported  dockside. 

For  these  reasons,  and  considering  the  state  of  the  art,  a  mix  of  off-line 
training,  CAI,  and  CMI,  is  viewed  as  the  preferred  system  design. 

C.  Third  Generation  Computer  Based  Instructional  Systems 

Two  types  of  system  configurations  have  been  developed  to  meet  the 
requirements  for  CAI  and  CMI.  A  major  goal  of  each  design  is  to  reduce  the 
operating  cost  of  instructional  terminals  to  within  the  cost  range  of  $0.35 
to  0.50  per  terminal  hour.  Technological  breakthroughs  have  made  it  possible 
to  locate  CRT  type  displays  at  great  distances  from  the  computer.  Other 
innovations  are  directed  toward  simplyfing  the  authoring  tasks  such  as 
course  entry  and  editing.  Each  configuration  will  probably  be  optimal  for 
certain  training  or  instructional  situations.  Prototype  examples  of  the 
systems  are  described  below. 

1.  The  TICCET  System 

Until  recently  no  computer  system  hardware  configurations  were 
designed  to  meet  the  unique  processing  requirements  of  CAI  or  computer  based 
instruction.  If  a  system  is  designed  specifically  for  computer  based  instruc¬ 
tion,  it  is  probable  that  very  high  performance  can  be  achieved  with  a  modest 
hardware  configuration.  Several  such  systems  are  being  designed.  A  pro¬ 
totype  example  is  the  TICCET  system  (Time-Shared,  Interactive,  Computer- 
Controlled  Educational  Television),  which  is  being  developed  under  an  NSF 
grant  by  MITRE  Corporation  in  conjunction  with  the  University  of  Texas  and 
Brigham  Young  University  (Stetten,  Morton,  and  Mayer,  1970).  MITRE  is 
responsible  for  the  hardware  configuration  and  computer  operating  software. 

A  technological  breakthrough  in  this  system  is  the  use  of  a  video  player 
to  refresh  the  TV  display.  There  is  also  the  possibility  of  using  cable  TV 
technology  as  the  communication  channel  from  computer  to  terminal.  The 
system  consists  mainly  of  off-the-shelf  components.  The  CPU  is  a  mini¬ 
computer  with  fast  cycle  time,  its  disks  are  fast  and  high-capacity  but  low 
cost,  and  data  channels  are  high  speed.  Terminal  equipment  consists  of 
ordinary  TV  monitors  with  keyboards.  For  electronics  training  these 
terminals  must  be  augmented  by  microfiche  projectors.  The  system  can  drive 
approximately  125  terminals.  The  cost  of  the  entire  system  including 
terminals  is  estimated  to  be  under  $350,000.  The  system  will  have  virtually 
all  the  capability  of  the  IBM  1500  system,  but  the  cost  per  terminal  hour 
would  be  reduced  by  a  factor  of  four  or  five  to  one.  The  system  could 
provide  the  required  computer  based  training  functions  for  many  applications 
of  a  mixed  mode  CAl/CMI  system. 

2.  The  PLATO  System 

A  large  time  sharing  system  can  be  made  cost  feasible  if  thousands 
of  terminals  can  be  connected  to  it.  The  problem  with  such  a  configuration 
is  that  a  large  number  of  users  usually  means  that  the  terminals  must  be 


distributed  over  a  large  area,  perhaps  hundreds  of  miles.  To  reduce  costs 
of  communication  lines,  either  clusters  of  terminals  must  have  sufficient 
memory  for  the  displays  in  the  cluster,  or  each  CRT  type  display  must  have 
its  own  display  memory.  PLATO  IV  is  a  large  time  sharing  system  which 
utilizes  a  plasma  tube  for  display  (Alpert  and  Bitzer,  1970).  The  image  is 
very  persistent,  remaining  on  the  display  until  written  over  by  new 
information.  The  physical  configuration  of  the  display  screen  permits  back 
projection  of  still  images.  Thus,  a  combination  of  random  access  still 
images  and  computer  generated  information  can  be  produced.  Responses  can 
be  made  either  by  keyboard  or  by  pointing  to  a  position  on  the  screen. 

This  terminal  is  a  very  versatile  and  powerful  device  for  instruction  as 
well  as  for  other  purposes. 

The  PLATO  system  is  being  evaluated  at  this  Laboratory  as  part  of 
the  1+3-03X  project. 


VI .  Conclusions  and  Recommendations 


Computer  assisted  instruction  provides  a  very  effective  method  of  train¬ 
ing.  CAI  students  scored  higher  than  class  instructed  students  on  both  the 
School  Examinations  and  the  Supplemental  Tests.  CAI  required  39%  to  less 
training  time  than  class  instruction.  Students  gave  high  ratings  to  their 
CAI  instruction  and  would  prefer  to  have  about  10%  to  80%  of  their  instruc¬ 
tion  via  CAI. 

Extended  experience  with  CAI  does  not  have  a  detrimental  effect.  On  the 
contrary,  long  term  CAI  was  found  to  produce  a  small  improvement  in  train¬ 
ing  time  and  an  increase  in  the  students*  preference  for  CAI.  Posttest 
performance  was  not  affected. 

Computer  based  training  methods  are  ideally  suited  to  the  development 
of  good  instruction  because  the  computer  performance  recording  and  process¬ 
ing  capabilities  make  it  possible  to  improve  instruction  to  any  required 
level.  Data  analysis  and  summary  programs  were  developed  in  this  project 
which  allowed  lesson  designers  to  identify  specific  course  weaknesses. 

Course  revisions  based  on  these  analyses  resulted  in  marked  improvements 
in  instruction. 

Although  CAI  showed  about  a  k5%>  time  savings  over  group  instruction, 
there  is  reason  to  believe  that  the  efficiency  of  CAI  can  be  improved  even 
further.  To  do  this,  increased  use  must  be  made  of  branching  technology  in 
order  to  provide  instruction  which  is  maximally  adaptive  to  individual 
differences.  A  number  of  advances  in  branching  technology  were  made  in 
this  project  and  incorporated  into  the  later  lessons.  The  result  was  an 
improvement  in  student  performance  over  the  previous  less  adaptable  CAI. 

In  addition,  it  was  found  that  student  option  was  just  as  effective  as 
program  control  of  training  by  means  of  pretests.  The  use  of  student 
option  simplifies  course  preparation.  Finally,  it  was  found  that  dual 
control  of  drill  and  practice  using  a  combination  of  program  control  and 
student  option  was  superior  to  strict  program  control.  Continued  research 
and  development  is  needed  to  further  improve  the  adaptability  of  instruction 
to  student  differences  in  ability,  knowledge  and  motivation. 
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The  most  costly  and  time  consuming  part  of  CAI  course  development  is 
the  initial  preparation  of  basic  instruction  —  computer  coding  and  programm¬ 
ing  of  these  materials  takes  far  less  effort.  CAI  shares  this  materials 
development  problem  with  all  forms  of  individualized  instruction. 

Earlier  CAI  systems  such  as  the  IBM  1500  system  can  compete  costwise 
with  classroom  instruction  in  some  applications.  However  they  are  generally 
not  cost  effective  in  comparison  to  individualized  instruction.  The  newer 
CAI  systems,  such  as  TICCET  and  PLATO  IV,  will  be  cost  effective  compared 
to  off-line  individualized  instruction,  and  will  provide  greatly  improved 
computer  based  training  capabilities. 

There  are  several  promising  new  areas  for  application  of  computer  based 
training  techniques.  These  include  shipboard  training,  use  of  CAI  in  multi- 
media  individualized  courses,  and  the  utilization  of  standard  CAI  terminals 
for  part  of  the  training  in  programs  which  normally  require  expensive  equip¬ 
ment  or  a  high  instructor-student  ratio. 
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